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Executive Summary 



This study examined high-performing after-school projects funded by The After- 
School Corporation (TASC), to determine what characteristics, if any, these projects 
shared. Evaluators reanalyzed student performance data collected during the multi-year 
evaluation of the TASC initiative to identify projects where the TASC after-school 
program was especially likely to have contributed to improvements in students’ academic 
achievement. Once these 10 projects were identified, evaluators visited each project to 
learn more about program structures and practices and whether the 10 projects shared 
common features. Interview and observational data revealed that, while these high- 
performing after-school projects were identified through their participants’ achievement 
gains, the projects did not primarily focus on academics. Across the 10 projects, 
evaluators found shared characteristics around programming, staffing, and support 
systems. These include: 

1. A broad array of enrichment opportunities: For many participants, the 
after-school project provided their first exposure to new learning 
opportunities in areas such as dance, music, art, and organized sports. 
Enrichment activities introduced participants to experiences that could 
spark interests and expand their goals for their own schooling, careers, and 
hobbies. 

2. Opportunities for skill building and mastery: Each after- school project 
created opportunities to build participants’ literacy skills through reading, 
story-telling, writing activities, and use of formal curricula, such as 
KidzLit and Passport to Success. In addition, these after-school projects 
integrated a focus on mastery into arts-based activities. Because arts 
activities involved practicing new skills in preparation for an exhibition or 
a performance, participants gained experience in practicing a skill to the 
point of mastery. 

3. Intentional relationship-building: This process began with each project 
fostering positive relationships with the host school, followed by steps to 
set a positive tone with staff through orientation, training, and 
establishment of participant norms. Throughout the year, the site 
coordinator worked on relationships with the project’s primary 
stakeholders through ongoing classroom-management training for staff, 
conflict resolution classes and team-building activities for participants, 
and regular communication with and the provision of support services to 
fa mi lies. 

4. A strong, experienced leader/manager supported by a trained and 
supervised staff: First and foremost, the site coordinators at these high 
performing projects brought with them experience in youth development 
and a strong connection to the community, the children, and the families 
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they served. Through orientations at the beginning of the project year, 
ongoing staff meetings and supervision, and consistent feedback on what 
worked and what didn’t work, all 10 site coordinators made efforts (and 
budgeted the time) to communicate and reinforce their vision of effective 
programming with their staff. 

5. The administrative, fiscal, and professional-development support of 
the sponsoring organization: The relationships between after-school 
projects and their sponsors built the foundation for the projects’ success 
and sustainability. In each partnership, the sponsor gave the site 
coordinator the autonomy and flexibility to manage the after-school 
project day-to-day, while providing administrative and fiscal support to 
the project. Each site coordinator was then able to use his or her expertise 
to select activities and make staffing decisions. 

These study findings can offer a guide to new and struggling after-school projects 
about program features that may be most important when developing or refining an after- 
school project. For established projects that find themselves under duress to increase 
their focus on academics or hire more academically focused staff, this study reinforces 
the viability of an after-school model that emphasizes a wide variety of compelling 
youth-oriented activities, a staff with diverse backgrounds and skills, an experienced site 
coordinator with strong ties to the community, the administrative and fiscal support of a 
committed sponsoring organization, and ongoing communication and relationship- 
building with the host school and participant families. 
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Shared Features of High-Performing After-School Programs: 
A Follow-Up to the TASC Evaluation 



Study Overview 



This study examined high-performing 
after-school projects funded by The After- 
School Corporation (TASC), to determine 
what characteristics, if any, these projects 
shared. Evaluators reanalyzed student 
performance data collected during the multi- 
year evaluation of the TASC initiative to 
identify projects where the after-school 
program was especially likely to have 
contributed to improvements in academic 
achievement. Once these 10 projects were 
identified, evaluators visited each project to 
learn more about program structures and 
practices and whether these characteristics 
were unique to individual projects or whether 
the 10 high-performing projects shared 
common characteristics. The findings from 
these inquiries are reported here. 

Selection of High-Performing Projects 

The study identified high-performing 
TASC projects in New York City based on 
changes in student achievement on New 
York’s statewide mathematics and English 
Eanguage Arts/reading (EEA) tests 
administered in grades 4 and 8 and on 
comparable citywide tests administered in 
grades 3, 5, 6, and 1 } Analyses measured 
changes in student achievement in math and 
EEA between the 2000-01 and 2001-02 



The After-School Corporation’s 
Service Delivery Model 

After-school services at each TASC project site are 
delivered by a nonprofit organization that has 
established a partnership with a public school that may 
serve any of grades K through 12. After-school activities 
are provided at the school, and all students enrolled in 
the host school are eligible to participate. The after- 
school program is in session from the end of each school 
day to approximately 6 p.m. Projects provide services 
free of charge. 

Each TASC project is led by a full-time, year-round, paid 
site coordinator who manages a diverse staff, including 
paraprofessional youth workers, arts specialists, certified 
teachers, and volunteers. Staff-student ratios are 
maintained at approximately 1:10 in the elementary 
school sites and slightly higher in the middle and high 
school sites. 

Educational enrichmentand homework help are 
provided in TASC projects, but projects also typically 
seek to nurture healthy youth development by 
encouraging positive adult and peer relationships and by 
exposing participants to a wide range of arts and 
recreational experiences. In addition, projects offer 
social supports such as health education, peer 
counseling, violence-prevention training, and 
opportunities to participate in community service, college 
preparation, and career training, as appropriate to the 
age of the youth served. 



school years, the most recent years for which data were available. 



The goal of the analysis was to identify projects where the TASC program was 
most likely to have contributed to improvements in student achievement. To improve the 



* The New York City Department of Education (DOE) provided the TASC evaluation with data from 
doe’s student-level administrative data files for school years 1997-98 through 2001-02, including student 
demographic characteristics, test scores, and school attendance. 
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focus on after-school program contributions, the analysis examined differences in gains 
between “active” participants (students who attended a TASC project for at least 60 days 
and 60 percent of the days it was possible for them to attend in 2001-02) and 
nonparticipants (students who attended the TASC host school in 2001-02 but never 
attended a TASC project in any year). Analyses further focused on active participants 
and nonparticipants who scored in performance level 1 (below basic) or performance 
level 2 (basic) in 2000-01, meaning that they performed below grade level. 

To determine the relationship between TASC participation and achievement gains 
in each of the 76 host schools in the study sample, the study subtracted the 12-month 
difference in average gains for nonparticipants from the 12-month difference in average 
gains for active participants within each school. Each project’s ELA and math gains 
were averaged into a single measure. Projects were then ranked based on the size of the 
difference in gains between active participants and nonparticipants, using the combined 
EEA/math measure. 

The top 20 projects based on these EEA/math rankings were determined to be the 
highest-performing TASC projects. To select the 10 study projects, evaluators asked 
TASC managers who work closely with the projects to comment on the current status and 
strengths of the top 20 projects. The TASC managers identified projects that were no 
longer operating in Spring 2005, or where significant changes had occurred in program 
quality, compliance with the TASC model, principal commitment to the program, or 
other factors that may have affected the project. Taking these comments into account, the 
10 most promising projects were invited to participate in the study, and they all agreed to 
do so. All 10 of the projects serve students in the elementary grades, and three of the 
projects also serve middle-grades students. 

Appendix A describes project selection in greater detail and lists the 10 study 
projects and the 10 projects from the top 20 that were not selected. 



Site Visits to High-Performing Projects 

A two-person study team visited each of the 10 study sites for two consecutive 
days in Spring 2005. Each site visit included a two-part interview with the site 
coordinator that focused on the project’s process and content features (e.g., goals, 
activities) and structural and institutional features (e.g., staffing, supervision, resources). 
In addition, the study team conducted structured 15-minute observations of after-school 
activities for six hours over the two-day visit, for a total of 14 to 20 independent 
observations per site. 

Site visitors used PSA’s Out-of-School Time Program Observation Instrument to 
capture and rate observable indicators of positive youth development. Developed for this 
study, the observation instrument builds on lessons learned through program observations 
in previous after-school studies, including the TASC evaluation (E.R. Reisner, R.N. 
White, C.R. Russell, & J. Birmingham, Building quality, scale, and ejfectiveness in after- 
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school programs: Summary of the TASC evaluation. Policy Studies Associates, 2004) and 
the Study of Promising Programs (D.L. Vandell, E.R. Reisner, B.B. Brown, K. 

Dadisman, D. Lee, & E.M. Pechman, The study of promising after-school programs: 
Examination of intermediate outcomes in Year 2, University of Wisconsin Center for 
Education Research, 2005). The instrument is also grounded in the youth development 
framework developed by the Community Network for Youth Development (M.A. 
Gambone & J.P. Connell, Community action framework for youth development. The 
Prevention Researcher, 11(2), 17-20, November 2004). 

The observation instrument allowed the study team to rate project activities on 
indicators addressing five key domains related to youth development. The five domains 
and their corresponding indicators are: 

1 . Y outh-directed relationship-building 

■ Youth are friendly to each other 

■ Youth show respect for one another 

■ Youth show positive affect to staff 

■ Youth are collaborative 

■ Youth assist one another 

2 . Y outh participation 

■ Youth are on-task 

■ Youth listen actively and attentively to peers and staff 

■ Youth contribute opinions, ideas, and/or concerns to discussions 

■ Youth have opportunities to make meaningful choices 

■ Youth take leadership responsibility /roles 

3. Staff-directed relationship-building 

■ Staff use positive behavior management techniques 

■ Staff are equitable and inclusive 

■ Staff show positive affect toward youth 

■ Staff attentively listen to and/or observe youth 

■ Staff encourage youth to share their ideas, opinions, and concerns 

■ Staff engage personally with youth, beyond task-related 
conversation 

4. Staff strategies for skill-building and mastery 

■ Staff communicate goals, purposes, expectations 

■ Staff verbally recognize youth’s efforts and accomplishments 

■ Staff assist youth without taking control 

■ Staff ask youth to expand upon their answers and ideas 

■ Staff challenge youth to move beyond their current level of 
competency 

■ Staff plan for/ask youth to work together 

■ Staff employ two or more teaching strategies 
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5. Activity content and structure 

■ The activity is well organized 

■ The activity involves the practice/a progression of skills 

■ The activity challenges students intellectually, creatively, and/or 
physically 

■ The activity requires analytic thinking 

Observers rated each indicator on a scale from 1 to 7, where a 1 meant that the 
indicator was not evident during the observation period, and a 7 meant that the indicator 
was highly evident and consistent. These ratings provided a systematic method for the 
study team to quantify its observations of the factors that contributed to high-performing 
after-school activities. A score of 5 meant either the exemplar was evident but 
inconsistent or that the desired behavior was generally present but not actively initiated 
and emphasized. For example, under youth-directed relationship-building, if youth 
relaxed together and enjoyed one another’s company but the activity did not involve a 
high level of socializing, the rating for “youth are friendly with each other” would be a 5. 
Likewise, under staff-directed relationships, if staff treated all youth in an inclusive 
manner but there was no need for staff to reengage an isolated child or group because 
every child was comfortable and included, then the rating for “staff are equitable and 
inclusive” would be a 5. 

The observation instrument was designed to include both indicators that were 
expected to be easily achieved in most after-school project activities, such as friendly 
relationships among youth, and indicators that were expected to occur less frequently, 
such as providing leadership roles for youth, or that were more challenging to implement, 
such as activities that require analytic thinking. To explore the presence of practices 
within each domain, the team created four scales that provided measures for examining 
the variation in each domain across types of activities. 

In total, site visitors conducted 173 independent observations at the 10 study sites. 
In addition to the independent observations, the two site visitors co-observed three 
observations in each study site to assess inter-rater reliability on the observation 
instrument.^ In total, 62 activities were co-observed, with an inter-relater reliability 
coefficient of 0.83. 

Further description of the observed activities and the technical details about each 
of the scales are presented in Appendix B. Appendix C presents the updated edition of 
the Out-of-School Time Program Observation Instrument used for this study. 



^ In addition, prior to the site visits, the study team participated in an intensive internal training designed to 
achieve high inter-rater reliability in the field. During this training, team members discussed and arrived at 
a common understanding of the definitions of the observation-instrument indicators. Team members also 
observed and rated videotaped after-school activity segments, then discussed their ratings. 
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Context for the Analysis of Shared Features 

This study was not designed to contrast the features of high-performing programs 
with the features of typical programs or low-performing programs. However, we know 
from several sources that the shared features described here are not necessarily typical of 
all TASC after-school programs. Or, more precisely, the features highlighted here are 
displayed by most TASC programs some of the time, but few TASC programs display all 
of the features all of the time. Evidence for this is presented in in the TASC evaluation 
(Reisner et ah, 2004) and in a substudy of the TASC evaluation that focused on programs 
serving middle-grades students (C.R. Russell & E.R. Reisner, Supporting social and 
cognitive growth among disadvantaged middle-grades students in TASC after-school 
projects. Policy Studies Associates, Inc., 2005). The following discussion cites 
applicable findings from those studies, as relevant. 

This point raises a larger issue about future research needs in the field of after- 
school programming. In general, more systematic information across large, diverse 
program samples is needed to illuminate the particular practices and approaches adopted 
by those after-school programs that achieve the greatest developmental gains for students. 



Shared Features 



Through interviews and observations at the 10 study sites, a consistent picture of 
structural and content features emerged. See Appendix D for more detail on each of the 
10 TASC projects. 



Programs Offered a Broad Array of Enrichment Activities 

These after-school projects balanced responsibilities for supporting youth 
academically with a commitment to engaging youth in high-quality enrichment 
activities. Activities in these projects occurred for almost three hours each school day in 
a safe environment that informally and constructively connected youth with a cross-age 

mix of peers and with positive adult role models. 
Project leaders felt strong external pressure to 
link their activities to school-day priorities and to 
ensure that participants finished at least some of 
their homework every day. But project leaders’ 
primary goals extended beyond participants’ 
academic learning. They were committed to 
exposing participants to a balanced array of new 
experiences that promoted healthy overall 
development. “We are focusing on the 
academics, but it’s not the be-all and end-all of 
the program,” one coordinator reported. “We’re 
[really] focusing more on the humanistic 



"My philosophy is to provide a safe 
environment in which children can 
develop and enhance a positive attitude 
towards academic, intellectual, cultural 
and artistic pursuits. We offer 
appropriate experiences and learn about 
social interactions...." 

Davon Russell 
Women's Housing and Economic 
Development Corporation (WHEDCO) 
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component,” another explained. When asked about their project goals, site coordinators 
said their mission was to create youth-centered, developmentally responsive activities 
first and academic support second. They sought to develop “holistic” learning 
opportunities that built “good and wholesome relationships” and “gave students a 
positive start in life.” 

Weekly schedules included homework help, project-based activities, arts, crafts, 
performing arts, and both structured and unstructured recreation. These high- 
performing after-school projects were safety zones where participants received 
homework help, explored new ideas and interests, and experienced long-term, supportive 
relationships with peers and adults. Occasionally-but not often-projects offered 
specialized academic support such as individual tutoring or small-group instruction. 
Academic and non-academic instruction was typically led by adult paraprofessionals or 
college students who were supervised by experienced program coordinators or certified 
teaching specialists. 

Arts and academic enrichment exposed participants to new experiences and 
gave them a sense of confidence to do things they had never tried before. “We’re 
interested in exposure,” a site coordinator explained. “We do not want to duplicate the 
day school.” For many participants, the after-school project provided their first exposure 
to learning opportunities such as dance, music, or a field trip outside their neighborhood. 
Site coordinators considered that their projects were helping participants “work toward 
[coordinator’s emphasis] mastery. . .learning through the process, and getting exposed to 
new concepts.” Thus, enrichment activities were designed to give participants the 
“broadest possible exposure” by introducing them to experiences that might spark their 
interests and expand their vision. Although they might not master every skill or content 
area, they were “exposed to everything,” and once they tried something, they could later 
choose to continue their involvement. 

Through sports and recreation activities, participants had time to run, play, 
interact in loud voices, and move freely with limited structure. These activities were the 
most informal part of the after-school project day and, for many participants, it was the 
first time every day that they could run freely in the gym or on the playground. Activity 
leaders sought a balance between structure and freedom so that participants could release 
a day’s worth of accumulated tension without getting out of control. In free-play 
activities, sports equipment was laid out and groups of participants chose their own 
games. In more structured sports, leaders set up the play area to give every participant an 
opportunity for practice and to play a key role. Activities included relay races, “double 
dutch” jump rope, volleyball, and other ball sports. Whether unstructured outdoor play 
activities or more structured activities in the gym, these recreational activities were 
friendly and fluid, giving staff and participants the time they needed to interact informally 
with one another in a relaxed, comfortable manner. This finding is consistent with 
analyses reported by Reisner et al. (2004) regarding higher levels of learning gains by 
participants in TASC projects that offered extensive opportunities for fitness, sports, and 
recreation. That report stated, “Evaluators speculate that these activities influenced 
participants both by drawing them into the after-school program and promoting high 
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attendance and also by providing the physical exercise needed for subsequent mental 
acuity” (page 43). 



Wide-Ranging Experiences Promoted Skiii-Buiiding and Mastery 

Each project approached its skills-focused activities differently, but all ensured 
that participants took part in three to six enrichment activities per week and that these 
activities involved them in group processes in which they developed a performance or 
product. Most projects hired specialists to lead several activities a week, which helped 
build an orientation toward skill-building and mastery in literacy, arts, sports, or 
community service activities. 

Participants built literacy skills through formal curricula. Each after-school 
program created opportunities to build participants’ literacy skills through reading, story- 
telling, and writing activities. Younger children participated in shared reading time, in 
which they described to one another the books they had chosen by reading short passages 
or describing the story in pictures. One group of pre-kindergarten and kindergarten 
participants maintained “munchkin journals,” which were notebooks in which they 
recorded the alphabet and words they were learning to use. Older participants wrote 
poetry, rehearsed plays, and designed newsletters and yearbooks. 

All 10 projects provided structure for some of these literacy activities by using 
formal curricula, such as the Developmental Studies Center’s KidzLit curriculum, 
Putamayo’s Passport to Success, and the Madison Square Garden Eiteracy Challenge. 
Participants in programs involved in the Eiteracy Challenge logged their reading 
activities and attended events at the Garden, which sometimes included a visit to the 
broadcasting booth and the opportunity to practice interviewing and broadcasting. 

In reviewing the 10 project schedules, science and mathematics activities were the 
least likely to be included in the schedule, although several programs included KidzMath 
and a limited number of “kitchen science” experiments. 

Homework help occurred in small groups, often managed by knowledgeable 
college students, with the assistance of teaching specialists or experienced after-school 
leaders. In many projects, high school students served as peer mentors and assistant 
activity leaders. At least four days per week (with Eriday often being the exception), 
project group leaders oversaw small groups of participants working on homework 
assignments. Eeaders tried to organize homework activities to engage participants when 
they were most alert and could benefit from assistance. Although many projects offered 
homework immediately after snack, the scheduling of homework for all groups at that 
time was not always feasible. One project worked around that problem by conducting 
homework activities all afternoon on the fifth floor of the school. Each group of after- 
school participants was scheduled for a 45-minute slot on the homework floor, where 
noise from other after-school activities could not be heard. 
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During homework sessions, group leaders were available to answer questions, 
help participants with assignments, and monitor participants in completing their 
homework, but rarely did they check participants’ homework for accuracy. One project 
motivated participants to be diligent in completing homework by offering time to play 
interactive logic and board games such as chess, scrabble, and checkers when participants 
finished their homework. 

At homework sessions, group leaders typically had books and academic games 
available that they distributed to participants who did not have homework or completed 
homework early. Although homework was largely directed by youth workers and not by 
certified teachers, site coordinators developed strategies for monitoring homework so that 
group leaders could turn to teachers if participants needed help that the group leader 
could not provide. Several site coordinators assigned certified teachers to circulate 
among groups during the homework period, while other coordinators directly monitored 
activity leaders’ lessons, recruited teachers from the partner school to mentor the program 
leaders, or asked their assistant coordinators to circulate during homework times to 
ensure an orderly working environment and to determine that participants were 
completing their assignments. 

Participants worked toward mastery in varied arts, recreation, literacy activities, 
and hands-on projects. In many instances, arts activities gave participants the 
opportunity to practice and master new skills in preparing for performances or 
exhibitions. Dance, music, and drama enabled participants to express themselves in new 
ways and were vehicles for creating cultural connections between the after-school project 
and its community. “I think for kids who are not academically successful, [arts activities] 
are another area in which they can be successful,” observed a project coordinator who is a 
former professional dancer. In her view, the learning that participants experience in arts 
and academics are “very parallel.” The value of experiential and hands-on learning after 
school in promoting cognitive growth was underscored in Russell and Reisner (2005), 
which compared the characteristics of after-school projects in which participants 
displayed high or low ratings on certain outcome measures. 

In the majority of activity observations across the projects, evaluators observed 
staff using one of three primary instructional strategies to promote skill-building and 
mastery. Observers examined the proportion of activities in which each indicator in the 
skill-building and mastery domain was rated as “evident” or higher (rating of 5 or higher 
on the seven-point scale). The following instructional strategies emerged as most evident 
in the activities that were observed: 

■ Program staff, either explicitly or implicitly, communicate the goals, 
purposes, and expectations of the activity in which youth are engaged (76 
percent of activities). 

■ Program staff assist youth without taking control of the activity, for 
example, by coaching or employing scaffolding techniques to help youth 
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gain a better understanding of a eoneept or eomplete an aetion on their 
own (66 pereent). 

■ Program staff verbally reeognize youths’ efforts and aeeomplishments (58 
pereent). 

Not all after-sehool aetivities that evaluators observed were intended to build 
speeifie skills. Many intentionally provided youth with less struetured opportunities to 
explore new experienees and develop relationships. Of the aetivities observed by 
evaluators, only 53 pereent were designed to build speeifie skills. Among those 
aetivities, there was even stronger evidenee of the three instruetional strategies deseribed 
above. In the skill-foeused aetivities, staff eommunieated aetivity goals elearly in 88 
pereent of these aetivities, assisted youth without taking eontrol in 79 pereent of the 
aetivities, and verbally eneouraged youth in 69 pereent. In addition, in a majority of 
these aetivities, observers rated the following instruetional strategies as a 5 or higher: 

■ Program staff ehallenge youth to move beyond their eurrent level of 
eompeteney through eonstruetive feedbaek (59 pereent of aetivities). 

■ Program staff employ two or more teaehing strategies to engage youth, 
ineluding a eombination of direet instruetion, eoaehing, modeling, 
demonstrating, or other strategies (55 pereent). 

In aetivities where instruetional strategies promoting skill-building and mastery 
were evident, staff members provided speeifie individual feedbaek and eneouragement to 
youth. For example, in a homework help aetivity for third-graders at the after-sehool 
projeet sponsored by Stanley M. Isaaes Neighborhood Center, the group leader regularly 
praised youth who were eoneentrating on their work, and reminded others to stay on task. 
The staff also emphasized aeeuraey, eheeking youths’ eompleted homework and 
providing feedbaek. Group leaders eoaehed individual students through math problems 
and eneouraged high standards in students’ expository writing. 

The use of instructional strategies that focused on skill-building and mastery 
varied by activity type. Evaluators eombined several indieators from the skill-building 
and mastery domain of the observation instrument to ereate a eontinuous seale seore, 
ranging from one to seven, to analyze variation in the emphasis on skill-building/mastery 
between different aetivities. This seale is deseribed in detail in Appendix B. Evaluators 
eondueted an ANOVA analysis to examine differenees in instruetional strategies, as 
measured by this seale, between types of aetivities. On average, staff in aeademie 
aetivities (seale seore of 4.27) and visual and performing arts aetivities (seore of 4.24) 
used skill-building instruetional strategies more frequently than did staff in sports 
aetivities (seore of 3.28). These differenees were statistieally signifieant. 

Programs provided numerous aeeess points to the arts. Eor example, the program 
sponsored by the Committee for Hispanie Children and Eamilies struetured the after- 
sehool experienee around theme-based arts aetivities. Throughout the year, eaeh group of 
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participants researched a theme related to the history and culture of New York City, then 
integrated their learning into a culminating performance through visual arts, drama, and 
music activities. A group of fifth-graders that investigated the role of a famous Puerto 
Rican leader wrote and choreographed a musical theater piece based on his life and 
work. Similarly, a group of kindergarteners at the same program designed a mural about 
the Bronx zoo and memorized a song about the animals they learned about. Elsewhere, 
performing artists came to after-school programs to teach various musical instruments 
(e.g., violin, drums, guitar) and dance forms (e.g., hip hop, African dance, capoeira). The 
content and focus of these activities depended on the resources available to the program 
sponsor, the schools, and the site coordinator. However, where sites could budget for 
professional artists and specialists to lead such activities, participants tended to benefit 
from a greater degree of skill-building and mastery-focused instruction. Observations 
showed that activities led by specialists had a mean scale score of 4.15 on the skill - 
building and mastery scale, which is a statistically significant difference from the mean 
scale score of 3.63 for activities that did not include these specialist staff. For example, 
one sponsor linked both the after- school program and the school with arts activities 
conducted by Dream Yard, a group of professional artists in New York City who go to 
urban classrooms to help children and youth learn to express, write, and perform their 
own stories. 

The projects demonstrated common elements within activity content and 
structure. In three-quarters of the activities observed, evaluators saw strong evidence 
that the activity was well organized (75 percent were rated 5 or higher). Evaluators also 
saw evidence that the activity challenged students intellectually, creatively, and/or 
physically in 58 percent of activities. 

An example of a well-organized and challenging activity that developed skills and 
required analytic thinking was a science enrichment activity observed at the program 
sponsored by the Committee for Hispanic Children and Families. The specialist leading 
the activity established a clear structure to integrate the development of science skills and 
deductive thinking into an exciting detective activity. Youth built on knowledge and 
skills presented at the beginning of the lesson and in previous lessons, and used analytic 
thinking to find clues. The specialist first gave an overview of fingerprint dusting, the 
focus of the day’s activity, then introduced participants to the materials they would be 
using, and walked them through the process of using their content knowledge to discover 
the clues for a crime they were solving, by matching printouts of fingerprints to those on 
pre-printed laminated cards. As participants worked, staff circulated around the room to 
pose questions, check on participants’ understanding, and help as needed. 

ANOVA analyses showed differing quality ratings based on activity type. 
Evaluators combined indicators measuring activity content and structure into a scale, and 
conducted ANOVA analyses to examine differences in scale scores by activity type. 
Analyses indicated that the average scale score for both academic enrichment activities 
(5.07) and arts activities (5.05) was significantly higher than for sports activities (3.60). 

In other words, sports activities were not as well organized and did not offer as many 
opportunities to develop specific skills or challenge participants’ thinking as did 
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academic enrichment and arts activities. In addition, activities that were categorized as 
intentionally skill-building had significantly higher scores on this scale than did activities 
that were not intended to be skill-building (5.27 compared to 3.33). That is, skill- 
building activities were consistently better organized, required more analytic thinking, 
and provided more appropriate challenges for participants than did non-skill-building 
activities. 



Intentional Relationship-Building Was a Primary Focus 

After-school projects had close working relationships with host schools. Strong 
partnerships with the leadership and staff at host schools helped determine whether a 
project ran smoothly or not. Essential ingredients of the most effective partnerships 
were: mutual respect between the project coordinator and the principal, shared teaching 
and paraprofessional staff members, appreciation that students benefited from the after- 
school experience, and flexibility among schools’ teaching, custodial, cafeteria, and 
security staff. “Everything is in the relationship with the school,” a site coordinator 
explained. “If they need something, we provide it,” another site coordinator offered. A 
principal made clear the symbiotic character of the after-school and school-day 
partnership: “We couldn’t exist without [the after-school project], but, then, the [after- 
school project] wouldn’t exist without us!” she explained. 

In the view of several coordinators, the absence of a rigid definition of project-to- 
school relationships enabled each project’s leader and staff to “invent solutions and create 
new ways of looking at the partnerships.” Among the study sites, seven had tight 
partnerships with their host schools. The three weaker partnerships were a result of 
recent changes either in school or project leadership, and coordinators were actively 
working to relieve any tensions. All projects scheduled regular planning times with the 
principals of their host schools, but there was no standard frequency of meetings or list of 
key individuals involved. Three project coordinators were on the leadership teams of 
their schools; another leader was about to become a member of her school’s leadership 
team. Six projects had a formal feedback mechanism for teachers to register concerns 
that arose when the after-school project used classrooms; other projects promoted 
informal connections among the teachers in whose rooms they worked and with the 
project leaders who managed groups. All projects elicited information from the school 
about homework assignments. The same types of close relationships between after- 
school staff and staff of the host schools were found by Russell and Reisner (2005) to 
promote high levels of school attendance on the part of after-school participants. 

Projects also aligned their expectations for youth behavior with the school’s 
expectations, allowing for greater informality in the after-school project. Hiring teachers 
and school aides from the school-day program ensured continuity between the norms of the 
school-day program and the after-school project. In addition, several projects called on 
school-based counselors to conduct group sessions on being friends, sexual awareness, 
healthy living styles, bullying, and reducing risk behavior. Because youth knew the 
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counselors, they could talk with those individuals about personal issues and express 
themselves freely. 

Among the most successful strategies undertaken by after-school projects to align 
academic priorities across the day-school and after-school projects were the following: 

■ Hiring a teacher from the day school whose job it was to keep the after- 
school project apprised of what the principal and teachers in the day 
school were doing 

■ Using literacy and mathematics materials that were used during the school 
day to help focus after-school academic support 

■ Observing in classrooms and talking informally with regular teachers 
about participants’ learning needs and behavior 

■ Pooling resources to hire arts and recreation specialists to work in the 
school day as well as in the after-school project 

■ Arranging for classroom libraries, manipulatives, and games to be 
available to support homework and other after-school academic activities 

■ Hiring school-day paraprofessional aides to coordinate the academic 
records of student progress between the school-day and after-school 
project 

■ Sharing the school’s parent liaison to facilitate connections between the 
school and families 

Finally, and significantly, joint celebrations and tangible appreciations were 
exchanged between the after-school and day-school projects. Projects allocated some of 
their own resources to provide the school staff with beginning-of-the year “care bags” 
and thank-you gifts and to host joint celebration and appreciation days. Teachers 
received materials they could use in their classroom, school staff received project 
T-shirts, and everyone school- wide was invited to come to all after-school performances, 
celebrations, and parties. 

Projects’ successes pivoted on the constructive relationships that project staff 
nurtured among participants. Coordinators measured their success by the “feeling of 
calm, respect, openness, and honesty” conveyed in activities and in informal exchanges, 
when “people just talk to each other.” They also judged their success by the degree to 
which participants acted and felt like a community, in the respect they gave one another, 
and in the relationships staff developed with families and community members. This 
perspective was echoed in findings reported by Russell and Reisner (2005); that study 
reported that, “in projects where coordinators reported offering a full roster of social 
development activities, including conflict resolution, peer discussion, and life skills 
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instruction, students were more likely to report a strong sense of community than were 
students in other projects” (page 14). 

Coordinators talked often with staff about “the importance of making every child and 
parent [know] they have a stake in what we do.” Coordinators described “re-teaching 
everyone” that when they come into the project, they “drop whatever baggage you’re 
coming in here with,” and treat people with decency, respect, and concern. They 
emphasized to their staff that it “is very important to talk to kids.” When behavioral issues 
arose, the first response was to “have a conversation and find out what’s going on” and then 
encourage youth to reflect on their behavior and its consequences with their project leaders, 
or with staff who had counseling training and could help sort out the root causes of 
hostilities. 

Managing behavioral tensions and teaching conflict resolution did not come easily 
to all staff, so site coordinators brought social workers and experienced youth specialists 
into projects in some instances to conduct relationship-building activities with staff and 
participants. One leader distributed a written description of “loveable things to say and 
do,” giving examples of words and actions that show participants that they were valued 
and respected. 

Periodic focused training, along with conversations during staff meetings, taught 
staff to conduct conflict-resolution activities and to strengthen peer relationships using 
written curricula such as Adventures in Peacemaking, Global Kids, and Resolving 
Conflict Peacefully. The curriculum materials suggested role-playing ideas and conflict 
scenarios with alternative endings for participants to work out, and activities from these 
curricula were included in each week’s schedule. In addition, group leaders used some of 
their activity time to: engage participants in discussions of behavioral norms; teach 
constructive ways of expressing anger, frustration, and dissatisfaction; and instill 
accountability for one’s own behavior. 

Staff were deliberate in their approaches to creating environments that fostered 
positive relationships. In more than three-quarters of the observations, evaluators rated 
the following indicators of staff relationships with youth at 5 or higher: 

■ Staff were equitable and inclusive with all youth, encouraging the 
participation of all and engaging any students who appeared to be isolated 
(90 percent of activities). 

■ Staff used positive behavior- management techniques, such as setting 
appropriate limits and communicating clear expectations for behavior (83 
percent). 

■ Staff showed positive affect toward all youth, using a caring tone and 
positive language (82 percent). 
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■ Staff attentively listened to and/or observed youth, paying attention as 
they completed a task and responding to what they said (82 percent). 

In one activity at the project sponsored by CAMBA, a trained social worker 
guided a group of seventh- and eighth-graders in a discussion about drug prevention. 

After acknowledging that it was a difficult topic, the staff member asked youth, “Why 
would a friend get involved in drugs? Why is it hard to stop them?” As the conversation 
proceeded, youth acknowledged they knew people who were trying drugs and that it was 
hard to resist peer pressure. Through her questioning, the staff member created an 
environment in which youth felt comfortable expressing their ideas and experiences on 
this difficult topic. She did not need to employ much active behavior management, 
because she had established a tone of respect in the activity. The leader worked to 
include all students in the discussion, and her respect for students was repaid by their 
openness. 

As with the other domains, evaluators combined indicators measuring staff-youth 
relationships into a scale to analyze patterns of variation. Academic enrichment activities 
had significantly higher scores on the staff-youth relationships scale than did homework 
help activities (4.99 scale score, compared with 4.16 scale score). In other words, in 
academic enrichment activities, staff tended to build more positive relationships with 
youth than they did in homework or tutoring sessions. 

Similarly, in the subset of activities that were skill focused (e.g., dance or drama 
rehearsals), observers saw significantly stronger staff-youth relationships than in 
activities that were not skill focused (e.g., games or open recreation time) (4.73 scale 
score, compared with 4.41 scale score). 

Participant norms and expectations were clearly established at the beginning of 
the project year. During project start-up, leaders worked in small groups to develop 
common expectations for participant interactions. They made their expectations clear 
and defined the consequences when those expectations were not met. New and returning 
participants worked together, and some projects hired project graduates who knew how to 
work with youth in ways that reflected the project’s values and commitment to 
establishing an atmosphere of mutual respect and collaboration. At the WHEDCO 
project, the year began with a week-long period in which the staff and returning 
participants worked through a series of community-building and “peacemaking” 
activities to “build relationships and make sure that expectations are clear” among staff 
and returning students. Using cooperative games from the Educators for Social 
Responsibility curriculum. Adventures in Peacemaking, the staff worked with returning 
students to determine “how our group is going to function from now until June,” as 
described by coordinator Davon Russell: 

Peer mediation is very important. [When there are conflicts,] we try to have the 
children address the situation or problem, and we try to offer them a different way of 
looking at things. . . . We encourage the staff to say, “Here’s another way of looking 
at it ... talk it out or let a teacher know.” When leaders speak with the kids, they 
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make a habit of asking, “What are your feelings?” Words need to be put to their 

feelings ... so I work with them on making a pact about taking constructive actions to 

avoid conflict as much as possible. 

In the activities observed across sites, youth demonstrated these positive 
relationships in various ways. In more than three-quarters of all the activities, observers 
saw evidence of positive youth participation, as measured by the following indicators: 

■ All or most youth were friendly to each other, demonstrating relaxed 
interactions with each other (rated 5 or higher in 93 percent of activities). 

■ All or most youth showed respect for one another, refraining from causing 
disruptions and considering each other’s viewpoint (86 percent of 
activities). 

■ All or most youth were on task (84 percent). 

■ All or most youth showed positive affect to staff, as demonstrated through 
friendly interactions (82 percent). 

■ All or most youth listened actively and attentively to peers and staff (76 
percent). 

In one activity at the project sponsored by Maspeth Town Hall, a small group of 
youth created a yearbook. While youth worked on the parts of the yearbook that most 
interested them, they also collaborated on other tasks in this ongoing activity. Three 
youth, using one computer, worked together on the yearbook’s layout, while another pair 
used a laptop to write a yearbook article. Youth were friendly and respectful of one 
another and of the teacher leading the activity, intently following the teacher’s 
demonstration of how to insert digital pictures into a template that the students had 
designed. 

As with the other domains, evaluators combined indicators measuring youth 
relationship-building and youth participation into a single scale. The analysis revealed 
statistically significant differences in the scale scores for arts activities, as compared with 
homework activities. On average, arts activities showed stronger youth relationships and 
participation than did homework activities (4.67 scale score, compared to 4.13 scale 
score). Homework activities were typically more structured and offered fewer informal 
opportunities for youth to work with each other. Analyses also showed significantly 
higher scale scores on the youth relationship-building and participation scale in activities 
that did not target academic skills. On average, activities that targeted physical, artistic, 
interpersonal, or other types of skills had a significantly higher scale score than activities 
that targeted academic skill-building (4.60, compared to 4.30). 

Families were included in the life of the project in various ways. Coordinators 
regarded the connection with fa mi lies to be vital to their success, although few families had 
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the discretionary time to become deeply involved in the after-school hours. Most projects 
were staffed with parent coordinators who were the primary point of contact between the 
family and the project; often, they were also employed by the school as its parent 
coordinator. 

At the beginning of the school year, project staff met with families in orientation 
sessions in which parents received information about program philosophy and approach, 
and they learned about the resources they could access through the project or through the 
program sponsor. Projects adopted policies that established a tone of welcome, respect, 
and inclusion. The policies also established project attendance requirements, behavioral 
expectations, and the consequences for participants who did not meet the behavioral 
expectations. 

Site coordinators and group leaders maintained regular contact with family 
members who picked children up at the end of the afternoon. This was an important time 
for sharing participants’ successes and news of forthcoming events. Newsletters and 
flyers went home to announce parenting workshops, ESL classes for adults, and 
community-oriented service activities and to alert parents to participant performances and 
projects. These activities gave parents with limited time the opportunity to participate in 
the project to the extent they could. Several projects periodically wrote reports on 
participants’ behavior and performance in activities either to parents or to the school, and 
this information was, in turn, included in teachers’ reports to parents. 

Families of children in after-school projects were also connected with the social- 
service resources of the program sponsors. The supportive resources available from 
sponsors was generally available to families at no cost, and the projects typically 
benefited from professional social workers, job counselors, and youth staff, often also at 
no extra cost. In some cases, the after-school project became the school’s connection to 
the sponsoring nonprofit organization’s services for families, even on behalf of students 
who were not enrolled in the after-school project. 



Programs Employed Strong Managers, Differentiated Staffing, and 
Supports for Line Staff 

These high-performing projects were led by site coordinators with five or more 
years of experience working for the grantee organization or for similar organizations; 
all 10 had strong ties to the communities where they worked. Some site coordinators 
had grown up in the same neighborhood as the host school; others had raised their 
children there or even sent them to the same host school. Virtually all of the site 
coordinators used such phrases as, “These are my children,” or “We’re family here,” 
when referring to their project participants, and a few had nieces and nephews who were 
enrolled in the after-school project. Most coordinators spoke Spanish fluently. 

Coordinators had a vision of what they were trying to accomplish, and they set 
goals and hired staff to achieve that vision. They understood that they were charged 
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with two pressing responsibilities: to promote positive cognitive development and to 
establish an emotional and developmental safety net that motivated participants to 
broaden their experiences and make a commitment to achieving personal and academic 
success. Leaders articulated their own vision of how to balance the potentially competing 
pressures of academic and social attentiveness, and they concentrated on convening a 
team of professionals, paraprofessionals, activities specialists, and community-based staff 
who shared a common belief in youth and in their capacity to achieve. “It’s about your 
vision and your dream being actualized, seeing it coming to fruition,” one site 
coordinator explained. 

Projects began their year with intensive mandatory orientation sessions, 
ranging in length from several days to a full work-week. At these meetings, staff 
members met and interacted with one another, learned project policies and procedures, 
contributed to planning the activity schedules, and discussed common behavior- 
management techniques. They typically received their assignments during orientation 
and made plans with their co-leaders. In four sites, staff were also introduced to the 
school-day curriculum, so that each group leader developed an awareness of the 
curriculum of the grade with which they would be working. 

Site coordinators led sessions cooperatively with their sponsoring organization’s 
mental health staff, host school principals, and assistant principals, and met with 
developers of curriculums they used. They distributed manuals that set out the project’s 
vision, defined standard approaches to motivating and disciplining youth, and explained 
policies and procedures for constructively managing student behavior and interactions 
with fa mi lies. 

Meetings focused on interactions with children and infused members of each 
site’s after-school team with the understanding “that when children are here they come 
first.” According to coordinators, it usually took some effort to bring activity leaders, 
security staff, principals, and teachers alike on board with the idea that children are at the 
front and center of all they do, but the common training experience was a crucial first 
step in this process. 

Training continued throughout the year in periodic full-staff meetings that took 
place after the project afternoon was over, and in day-long in-service sessions that 
focused on using curricula or managing children with developmental sensitivity. 

Seven of the 10 projects conducted regular, paid staff meetings. Four projects conducted 
one- to two-hour meetings each week; three projects met about once a month. The 
remaining three projects held staff meetings on an as-needed basis only. 

Scheduling meetings with part-time staff was never easy, but project coordinators 
in these sites met the challenge by extending the work day one hour, reducing the 
program day for students by an hour, or holding meetings on days when school-and 
therefore the project-was not in session. Weekly meetings varied in length from a half 
hour to two hours; sites without regular meetings emphasized direct one-on-one 
mentoring between the site coordinators and activity leaders; in several cases, the site 
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coordinator and the assistant coordinators conducted periodic classroom observations. 
Two sites supplemented their staff meetings with brief memos or more extended 
newsletters. One site conducted their meetings on Friday afternoons, making them “rap” 
sessions and extending them to include a social hour and dinner. 

Most sites used staff meeting time to discuss logistics or scheduling changes and 
to exchange ideas on how to support specific participants more fully or to convey 
messages from families or from the school-day staff to the program staff. Often, in 
addition to logistical matters, internal staff or external consultants offered specialized 
training sessions. 

Site coordinators valued the contribution of teachers from host schools who 
would periodically conduct strategy workshops for staff. “There’s nobody better to teach 
my staff how to do homework and math literacy help than the literacy and math coaches 
for the teachers at the school,” reported one coordinator. In addition to informing project 
staff about what was happening during the school day, they brought feedback from the 
after-school staff sessions to their sessions with their colleagues. 

Mentoring, guidance by managers, lesson planning and open dialogues among 
staff continued to strengthen professionalism. Long-time staff mentored new staff, and 
older staff mentored younger staff to develop common strategies for managing behavior 
positively and for motivating participants through hands-on experiences, games, and 
projects. In sites with assistant coordinators or certified teaching personnel, trained staff 
were paired with the less-experienced staff to learn how to plan and write lessons and to 
work with teaching curricula such as KidzLit and KidzMath, Junior Achievement, and 
Adventures in Peacemaking. 

About half of the projects required leaders to plan and submit lesson plans. In 
addition, in many projects site coordinators and classroom teachers helped the 
paraprofessional leaders develop plans. Lesson plans, when they were required, were 
simple outlines, and did not require staff to go into great detail. “I don’t ask staff to make 
it very intricate,” one site coordinator reported. “Just something that will get them from 
point A to point Z. I try to have them write out a realistic idea of what is going to 
happen.” The TASC evaluation found that requirements for activity or lesson plans were 
associated with higher test-score gains (Reisner et ah, 2004). The report also stated, 
“Evaluators speculate that the preparation and review of written plans occurred mainly in 
projects in which student learning was a high priority” (pages 43-44). 

Additional mentoring occurred through both formal and informal evaluations. Six 
projects conducted routine one-on-one staff evaluations at least once a year; in four 
projects evaluations were less systematic. Two projects asked staff to submit a self- 
evaluation checklist to the site coordinator every six to eight weeks so the staff and 
coordinator could discuss strengths and weaknesses. Other projects had unit leaders and 
teachers observe activities and provide either structured or informal feedback to the 
activity leaders they observed, but not to the project coordinator. All of the projects 
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conducted year-end staff assessments in preparation for rehiring and planning training for 
the following year. 

Staff also accessed training from TASC and others. In addition to bringing 
trainers on site to customize staff training, site coordinators were alert to training sessions 
offered by external vendors, and they notified staff who would benefit from such training. 
When staff members volunteered to attend these outside training sessions, they were 
usually paid an hourly stipend, and several coordinators asked participants to report back 

In New York, training courses are 
available from TASC and others on 
numerous topics, including “adult 
fundamentals,” which provides “basic after- 
school training,” such as activity planning, 
managing group and individual behavior, 
and responding to the developmental needs 
of participants. Different levels of courses 
are available to staff with different levels of 
preparation, so that courses for experienced 
professionals differ from courses for high 
school and college youth workers. TASC 
also offers in-service programs about how to 
support a range of academic and 
nonacademic learning through literacy, 
mathematics, and other traditional school subjects. Curriculum developers conduct 
periodic intensive in-service programs on using their curricula, and site coordinators 
encourage staff to attend these programs because they tend to be “more of an 
informational and inspirational type of training” than the briefer training sessions that 
were integrated into the project’s staff development routine. 

Leaders used informal networks to identify potential new staff Coordinators 
reported a very high staff return rate, as much as 75 to 80 percent per year. They 
believed the care they took in hiring staff, despite part-time hours and low wages, 
contributed to high retention rates. Site coordinators managed their own hiring, and they 
turned to members of their eommunity, including paraprofessionals who worked in their 
partner schools and community people who worked with the program sponsors, to 
identify good matches for project openings as they occurred. Three projects created 
training opportunities that allowed them to “grow our own” staff by giving older students 
a chanee to work within the project during their high school years, eventually hiring them 
after they were 16 years old and eligible for paid employment. 

Coordinators were intuitive about which staff were the right fit for their project. 

As one site coordinator said, “I can read between the lines when I am hiring someone. . . . 

[I look for] passionate, committed young people. The way they walk into the building, 
the way they look at the kids, if they notice the work on the wall.” They also favored 



to their colleagues about what they learned 



The key to running the project well is to be 
nurturing to your staff. Have your staff know 
you have a vested interest in what happens to 
them. I want my staff to use this as a growing 
experience... . They know that [I] care about 
what goes on in their lives. I call them 'my 
kids.'... . I feel like they're my kids. I celebrate 
with them when they do something good; I 
worry when something bad happens: and I 
grieve for them when they [face a serious loss]." 

J an Vasquez 
Sports and Arts in Schools Foundation 
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staff who had special interests and talents, such as coaching, videography, arts and crafts, 
and directing choirs or plays. Significantly, they looked for “high-energy people who 
have fun with kids.” Referring to the important balance between informality and 
positive, disciplined leadership, one site coordinator said, “Not everyone gets it.” 

Coordinators interviewed prospective hires in depth, and they relied on the 
recommendations of staff they respected. Two projects also asked potential new hires to 
spend a day with the project before either the project or the new staff member committed 
to one another. Seven coordinators had adopted formal application and interview 
processes; the three others recruited and hired informally, and were always on the lookout 
for candidates who “see the vision” and are a good match for their project. They said that 
the success of these strategies depended on identifying community-based staff and, once 
they came on board, providing them with continuous training. The two elements, 
community staffing and training, worked together: “When you have one and not the 
other, [something is missing,]” said a coordinator. 

Site coordinators regularly described their work with children as a calling, and 
they sought personnel who came with commitment to hard work and a willingness to 
grow. One site coordinator began her annual staff orientation by addressing her staff’s 
sense of calling to work in education and youth development: “If you think you want to 
be a teacher, you will either know it or be cured of it before leaving here; and, believe 
me, we have sent on a lot of teachers and I’ve cured a few people of the illusion that they 
wanted to be teachers.” 

Differentiated staffing structures reflected project priorities, available talent, 
and relationships with host schools and program sponsors. Site coordinators, who had 
many years of experience within their after-school project, knew how to take advantage 
of the services offered by their nonprofit sponsor, and how to forge connections with 
leaders and teachers in their host schools. Projects worked from a “strength-based 
perspective” by convening teams to implement their dual academic and humanistic 
vision. Each project was led by a small management team, typically including the site 
coordinator and one or two assistants, which had the independence and flexibility to hire 
and train staff and to manage day-to-day operations and activities. Staff based in the 
sponsoring nonprofit organization worked flexibly with the site coordinators, providing 
the administrative structure for hiring, evaluating, and paying staff and, in some cases, for 
providing training and professional development. 

Site coordinators supplemented their own skills as leaders with assistants and 
professional educators who oversaw project content and quality, supported activity 
implementation, and, in some cases, helped mentor staff as they planned and carried out 
structured lessons. Sites recruited two to five certified teachers — typically with 
specializations in literacy, math, or science instruction — who had demonstrated that they 
could work informally with paraprofessional and youth staff and with participants, many 
of whom were struggling academically. In addition, staff structures included either 
parent coordinators or social workers who maintained contact with fa mi lies and 
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connected fa mi lies to social services resources that were available to the project through 
the project sponsor. 

Because few activity leaders or their assistants were experienced teachers or 
professionals, site coordinators recruited certified teachers to work side-by-side with the 
activity staff. Certified teachers might write lesson plans and demonstrate how these 
could be used to develop academic activities that actively engaged participants, or they 
might offer demonstration lessons or lead activities with the regular group leader 
assisting. One program director coordinated with the school-day literacy coaches to 
identify materials that the after-school project could use that were aligned with 
instructional materials employed during the school day. This director also hired a 
certified language arts teacher to direct language arts skill-building activities for 
kindergarten through second grade while providing academic enrichment, test-taking 
skill-building, and literacy development for older students. Two projects identified 
school-day science or math coordinators to “link with the school day.” Another project 
recruited three “skill developers” from outside of the school to supplement academic 
activities offered by project leaders. Several projects hired certified teachers to circulate 
around work areas during homework time to help provide support (e.g., classroom 
management, instructional) and supervision to leaders. 

Visiting artists and recreation specialists brought expertise in arts and sports to 
vary and strengthen the quality of project offerings. Every after- school project in the 
study offered three to four opportunities a week for students to participate in varied arts 
and sports activities. In some projects, staff with special skills in these areas were hired 
as group leaders who took on special initiatives within the project. Other projects hired 
teachers from the regular school day who were looking for opportunities to teach 
something new and creative, and some made connections with past participants and hired 
them to assist in the project. Teachers from the school hosting the project of the Sports 
and Arts in Schools Foundation became impresarios and led theater productions each 
year. Other projects contracted with professional artists or recreation specialists from 
community arts and recreation organizations. Specialists were often accomplished 
performers who brought both a discipline and a passion for their work. Arts specialists 
included musicians, visual artists, actors, dancers, while recreation specialists included 
skilled athletes who were coaches and players on community teams. 

Projects provided strong support for line staff Salaries in after-school projects 
are modest, and quality performance and success with youth are rarely rewarded 
financially. Most staff in these projects were students or school-based paraprofessionals. 
Salaries began as low as $7 to $10 an hour for high school students and inexperienced 
college students; school aides, paraprofessionals, and older college students were 
typically paid $10 to $12 per hour; arts specialists (who were not certified teachers) 
earned as much as $20 to $30 per hour. Certified teachers, whose contract allowed them 
to earn an after-school per-session rate of upwards of $35 per hour, sometimes 
volunteered to work for slightly less (e.g., $25 to $30 per hour). 
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Projects compensated for low salaries by providing opportunities for staff to 
receive mentoring and in-service training and by offering a collegial community. As a 
group, they took communication very seriously. Each coordinator found a unique 
communication strategy that supported the staff members in his or her setting, but some 
common themes arose across projects. Leaders began the year with start-up training 
sessions and regular staff meetings that included all staff. They also conducted periodic 
job-embedded staff development, mentoring, and off-site training opportunities. 

As with other features of effective after-school programming, both formal and 
informal structures and the flexibility to adapt to rapidly changing circumstances 
developed line staff skills. “We spend as much time on our employees as on our kids,” a 
site coordinator explained. “They feel like they’re growing and gaining, so it’s not just 
an after-school job.” 

Site coordinators relied on good communication and collegiality to maintain the 
quality of their projects. In the face of low salaries, small salary increases, and 
professional advancement only with additional schooling, site coordinators relied on the 
power of their relationships to maintain stable, committed staffs. Communicating praise 
and encouragement went a long way toward building collegiality and commitment to the 
project. “In our meetings, I say thank you all the time. I always begin by saying, ‘Thank 
you for everything you do,”’ said one coordinator. Another coordinator reported: “I tell 
[the staff], ‘You’re doing much more work than I can ever pay you for,’ I’m very up- 
front with them.” 

Coordinators included staff in project planning in a number of ways. In addition 
to the planning done in staff meetings, some site leaders formally and informally 
surveyed staff so they could anonymously report their thinking about project quality 
issues. A site coordinator said, “I take the time to put things in writing and to have 
people respond, and then we have a relationship where we’re all doing something 
[together]. I respect their opinions... and everybody feels like they have something to do 
with the project’s success.” 

Coordinators also used social events to strengthen informal ties. They celebrated 
birthdays and special occasions with cakes and gifts. They periodically hosted pot-luck 
meals for staff or convened over pizza, and sponsors included after-school staff in 
neighborhood celebrations and holiday parties. 

Coordinators made clear that their projects worked best when they functioned 
“like a family.” They know there is much they must do to maintain their project quality. 
But all the management and training go only as far as the quality of relationships they 
have been able to nurture within their staffs and among staff, students, schools, and 
families. Finally, program strength lies in the success of the team and in its capacity to 
function effectively together. “We’re very proud to work with the project,” a site 
coordinator reported. “My organization, the [advisory] committee-they know that we 
have been accomplishing a lot. But they know it’s not my project. It’s teamwork.” 
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Partner Organizations Provided Support to Project Leaders and 
Participants 

Sponsoring nonprofit organizations actively supported after-school projects 
across all sectors of their operations. The relationships between after-school projects 
and their partners built the foundations of the projects’ success and sustainability. Site 
coordinators at the high-performing projects enjoyed autonomy to make programming 
decisions, and there was mutual respect between them and their supervisors. Along with 
the relationship with the host school, this partnership “sets the stage for everything else,” 
one site coordinator noted. “I spend a lot of energy doing the right thing to keep these 
relationships healthy.” 

In these partnerships, the sponsoring organization gave the site coordinator the 
autonomy and flexibility to manage the after- school project day to day, while providing 
administrative and fiscal support to the project. Site coordinators were then able to use 
their expertise in selecting activities and in making staffing decisions. Regular 
communication between the sponsor and the site coordinator kept both parties up to date. 
This communication often took the form of weekly updates, monthly reports, meetings, 
or telephone conversations. 

In addition to linking fa mi lies and participants with its services, the sponsor 
provided resources for the project itself in the form of staff professional development. 
Some sponsors ran more than one after-school project, and site coordinators drew on the 
experiences of their colleagues to solve problems within their own projects. Other 
sponsors developed extensive, in-house training that all new staff attended at the 
beginning of the year. Some experienced site coordinators shared their own knowledge 
by leading professional development sessions within and outside the sponsor 
organization. 

Strong financial and managerial support from the sponsors freed site coordinators 
to concentrate on creating thriving after-school projects. The sponsors identified and 
secured resources to ensure the sustainability of the after-school project. These projects 
also benefited from the shared funding streams available to them through the sponsors’ 
resources. For instance, while an after-school project awaited a grant payment, its 
sponsoring organization used other fiscal lines to pay for staff salaries and needed 
supplies. Site coordinators at these high-performing sites described their sponsoring 
organization as creative and flexible in locating financial resources for the projects. 



Conclusions 

While these high-performing after-school projects were identified through their 
participants’ gains on math and ELA state and city wide tests, the projects did not share a 
targeted focus on academics. Rather, projects contributed to their participants’ learning 
gains by providing a broad base of opportunities and supports. Across the projects. 
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shared characteristics around programming, staffing, and support systems emerged. 
These include: 



1. A broad array of enrichment opportunities: For many participants, the 
after-school project provided their first exposure to new learning 
opportunities in areas such as dance, music, art, and organized sports. 
Enrichment activities introduced participants to experiences that could 
spark interests and expand their goals for their own schooling, careers, and 
hobbies. 

2. Opportunities for skill building and mastery: Each after- school project 
created opportunities to build participants’ literacy skills through reading, 
story-telling, writing activities, and use of formal curricula, such as 
KidzLit and Passport to Success. In addition, these after-school projects 
integrated a focus on mastery into arts-based activities. Because arts 
activities involved practicing new skills in preparation for an exhibition or 
a performance, participants gained experience in practicing a skill to the 
point of mastery. 

3. Intentional relationship-building: This process began with each project 
fostering positive relationships with the host school, followed by steps to 
set a positive tone with staff through orientation, training, and 
establishment of participant norms. Throughout the year, the site 
coordinator worked on relationships with the project’s primary 
stakeholders through ongoing classroom-management training for staff, 
conflict resolution classes and team-building activities for participants, 
and regular communication with and the provision of support services to 
fa mi lies. 

4. A strong, experienced leader/manager supported by a trained and 
supervised staff: Eirst and foremost, the site coordinators at these high 
performing projects brought with them experience in youth development 
and a strong connection to the community, the children, and the families 
they served. Through orientations at the beginning of the project year, 
ongoing staff meetings and supervision, and consistent feedback on what 
worked and what didn’t work, all 10 site coordinators made efforts (and 
budgeted the time) to communicate and reinforce their vision of effective 
programming with their staff. 

5. The administrative, fiscal, and professional-development support of 
the sponsoring organization: The relationships between after-school 
projects and their sponsors built the foundation for the projects’ success 
and sustainability. In each partnership, the sponsor gave the site 
coordinator the autonomy and flexibility to manage the after-school 
project day-to-day, while providing administrative and fiscal support to 
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the project. Each site coordinator was then able to use his or her expertise 
to select activities and make staffing decisions. 

These study findings can offer a guide to new and struggling after-school projects 
about program features that may be most important in developing or refining an after- 
school project. For established projects that find themselves under duress to increase 
their focus on academics or to hire more academically focused staff, this study reinforces 
the viability of the TASC model: together, a wide variety of compelling youth-oriented 
activities, a staff with diverse backgrounds and skills, an experienced site coordinator 
with strong ties to the community, the administrative and fiscal support of a sponsoring 
organization, and constant communication and relationship-building with the host school 
and participant families can contribute to gains in school performance for after-school 
participants. 
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Appendix A 

Selection of High-Performing Projects 



The study identified high-performing TASC projects based on changes in student 
achievement on the statewide mathematics and English Language Arts/reading (ELA) 
tests administered in grades 4 and 8 and on the comparable citywide tests administered in 
New York City in grades 3, 5, 6, and 1 } Individual gains in reading and math scores 
were aggregated to the project level. Project-level gains in ELA and math scores were 
then averaged and ranked. 

Analyses focused on changes in student achievement in math and in ELA between 
the 2000-01 and 2001-02 school years, the most recent years for which data were 
available. Because the scaling of student achievement test scores for the New York City 
and state tests does not follow a regular progression across grade levels, and does not 
provide a standard for the expected gain between grade levels, the study first transformed 
each student’s scale score at each grade level into a standardized scale score that could be 
used to compare students’ change in achievement across grades.^ Using this method, 
differences in the standardized scale scores across grade levels are expressed as 
differences in the proportion of possible scale-score points that a student earns in one 
grade level compared with the proportion earned at the next. 

The goal of this analysis was to identify specific projects where the TASC 
program was most likely to have contributed to improvements in student achievement. 
Therefore, the analysis examined differences in gains between active participants 
(students who attended a TASC project for at least 60 days and 60 percent of the days it 
was possible for them to attend in 2001-02) and nonparticipants (students who attended 
the TASC host school in 2001-02 but never attended a TASC project in any year). Eor 
each site, analyses focused on active participants and nonparticipants who scored in 
performance level 1 (below basic) or performance level 2 (basic) in 2000-01, meaning 
that they were performing below grade level and hence most in need of academic 
improvement. The analysis was also limited to TASC sites with at least 25 active 
participants and 25 nonparticipants for whom test score data were available in both 2000- 
01 and 2001-02, and who therefore could be included in the change analysis. Together, 
these constraints helped to ensure that the analysis of gains compared nonparticipants and 
active participants with similar initial achievement levels, and helped to increase the 



*. The New York City Department of Education (DOE) provided the TASC evaluation with data from 
doe’s student-level administrative data files for school years 1997-98 through 2001-02, including student 
demographic characteristics, test scores, and school attendance. 

^ The approach used in this study was to standardize the scale scores across grades, so that the range of 
possible test scores extended from 0 to 100 at each grade level and the mid-point of the possible scale 
scores for each grade level were always 50. The formula used to transform each student’s scale score at 
each grade level into a standardized score is { [(scale score-minimum possible scale score)/(maximum 
possible scale score-minimum possible scale score)] x 100} 
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chances of identifying TASC projects that contributed to improved achievement for 
participants who attended regularly. 

For each student in the 76 TASC host schools who met these requirements, the 
study computed changes in performance on the city and state ELA and math achievement 
tests by subtracting the standardized scale score in 2000-01 from his or her standardized 
scale score in 2001-02. That is, for an estimate of the changes in student performance, the 
study computed the difference in the proportion of possible scale-score points that a 
student earned in one grade level compared with the proportion of possible points earned 
at the next. Student-level changes in ELA and in math were then aggregated to the site 
level separately for active participants and for nonparticipants. To determine the relative 
effectiveness of TASC participation on achievement gains in each of the 76 schools, the 
study subtracted average gains for nonparticipants from average gains for active 
participants within each school. Projects were then ranked based on the size of the 
difference in gains between active participants and nonparticipants, in ELA and in math. 
In order to identify the overall high-performing TASC projects for the purposes of this 
study, each site’s ELA and math gains in rank were averaged into a single ranking. 

The top 20 projects based on these average ELA/math rankings were determined 
to be the highest-performing TASC projects. To select the 10 study sites, the study asked 
TASC managers who work closely with the projects to comment on the current status and 
strengths of the 20 projects. The TASC managers identified projects that were no longer 
operating in Spring 2005, or had experienced significant changes in program quality, 
compliance with the TASC model, principal commitment to the program, or other factors 
likely to have affected the program. Taking these comments into account, evaluators 
invited the 10 most promising projects to participate in the study. All 10 of the selected 
study projects serve students in the elementary grades, and three of the projects also serve 
middle-grades students. 

The following exhibits describe the 10 study projects (Exhibit A-1) and the 10 
projects that were identified as high-performing but not included in the study 
(Exhibit A- 2). 
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Exhibit A-1 

High-Performing Projects Seiected for the Study 



Nonprofit Sponsor 


School 


Borough 


Grade 

levels 

served 


Church Avenue Merchants Block 
Association (CAMBA) 


PS/IS 25 


Brooklyn 


K-8 


Committee for Hispanic Children and 
Families (CHCF) 


PS/MS 279 


Bronx 


K-8 


Cypress Hills Neighborhood 
Development Center 


PS 7 


Brooklyn 


K-4 


East Side Settlement House 


PS 220 


Bronx 


K-8 


Maspeth Town Hall 


PS 229 


Queens 


K-6 


Ralph Lincoln Service Center 


PS 12 


Brooklyn 


K-8 


Samuel Field Y 


PS 115Q 


Queens 


K-5 


Sports and Arts in Schools 
Foundation 


PS 122 


Queens 


K-8 


Stanley M. Isaacs Neighborhood 
Center 


PS 1 98/ 
PS 77 


Manhattan 


K-5 


Women’s Housing and Economic 
Development Corporation (WHEDCO) 


CES218 


Bronx 


K-8 



Exhibit A-2 

High-Performing Projects Not Inciuded in the Study 



Nonprofit Sponsor 


School 


Borough 


Grade 
levels 
served by 
school 


ASPIRA 


MS 143X 


Bronx 


6-8 


Brooklyn Chinese-American 
Association 


PS 160 


Brooklyn 


PreK-5 


Children’s Aid Society 


PS 152 


Manhattan 


K-5 


Children’s Aid Society* 


PS 146 


Bronx 


K-5 


Church Avenue Merchants Block 
Association* 


PS 109 


Brooklyn 


PreK-5 


East Side House 


PS 43 


Bronx 


K-6 


Greater Ridgewood Youth Council 


PS 71 


Queens 


K-5 


New York University School of 
Education, Metro Center* 


PS 123 


Manhattan 


K-6 


Police Athletic League 


PS 80 


Staten 

Island 


K-12 


Sports and Arts in Schools 
Foundation 


IS 125Q 


Queens 


5-8 


YMCA of Greater New York/Brooklyn 
Central YMCA/Henry Street 
Settlement 


PS 20 


Brooklyn 


K-6 



* Project no longer open 
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Appendix B 

Summary of Activity Observations and Domain Scales 



Activity Content 

Site visitors intentionally observed various projeet activities in each study site in order to 
capture a diversity of activity structures and instructional approaches. To the extent possible, 
given the schedule of project activities in each site, site visitors were encouraged to observe at 
least two activities of each of the following types: 

■ Homework help, tutoring, and test-preparation activities 

■ Content-based academic enrichment activities (e.g., literacy, numeracy, science, 
or newspaper) 

■ Arts activities (e.g., visual and performing arts, including dance and drama) 

■ Sports activities (e.g., open sports/movement, and structured athletics, including 
martial arts) 

■ Other enrichment activities (e.g., outing club, girls’ group, investment club, 
college club) 

In total, site visitors conducted 173 independent observations across the 10 study sites. 
Overall, visual and performing arts activities were most frequently observed (30 percent of total 
observations), followed by homework help (22 percent), academic enrichment activities (20 
percent), and sports activities (12 percent). The specific types of activities observed are 
illustrated in Exhibit B-1. 



Exhibit B-1 

Types of Activities Observed, Spring 2005 



Activity 


Percent of Observations 
(N=173) 


Homework help/tutorinq 


23 


Dance, music, drama 


20 


Academic activities 


13 


Arts and crafts 


12 


Open, unstructured time 


12 


Sports-pi ay ing physicai games 


11 


Other activity type 


9 


Story reading/listening 


8 


Teacher-assigned learning games 


5 


Clean-up/transition 


4 


Cultural awareness clubs/projects 


3 


Sports-practicing/learning a skill 


2 


Community service 


1 


College/career development 


1 


Computer skill development 


1 



Note: Observations could include more than one activity type. Therefore, the total percent exceeds 
100 . 

Exhibit reads: Homework help or tutoring activities were present in 23 percent of observations 
conducted in the study. 
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Of the activities observed, 36 percent were classified as targeting specific academic 
skills, such as reading and literacy skills, and mathematics/numeracy skills. 



Activity Context 

The study observed activities serving a wide range of grades, and grade level was the 
primary means by which youth were grouped for activities (75 percent of observations). In 17 
percent of the observed activities, youth selected to attend the activity based on their interest. 
The grade-level distribution of observations is shown in Exhibit B-2. 

Exhibit B-2 

Grade Leveis Observed, Spring 2005 



Grade Level 


Percent of Observations 

(N=173) 


Kindergarten 


14 


1 


21 


2 


32 


3 


24 


4 


25 


5 


27 


6 


14 


7 


8 


8 


7 



Youth from more than one grade could have been present during an 
activity. Therefere, the total percent exceeds 100. 



Exhibit reads: Kindergarteners were present in 14 percent ef activities 
observed. 

Overall, the staff-youth ratio was extremely low in the activities observed. On average, 
across the activities observed in each of the 10 study sites, there were eight girls and six boys 
present in an activity. Evaluators observed an average of two staff members in each activity, 
leading to an average staff-youth ratio of one to six. The activities observed were staffed by 
combinations of high school and college students, other adults, certified teachers, and specialists 
such as musicians and dancers. Most often, college students or other adults were present (in 51 
percent and 54 percent of activities, respectively.) High school students were present in one 
third of the activities (33 percent), while specialists instructed students in 25 percent of 
activities. Eleven percent of activities were supervised by certified teachers. (Percentages do not 
add to 100 because more than one type of staff was present in most activities observed.) 

The environmental context was consistently appropriate to the activities observed. In 97 
percent of observations, evaluators observed a level of supervision appropriate to the activity and 
age of the students. Activities most frequently took place in classrooms. In addition, evaluators 
observed activities that took advantage of a range of facilities in the host school, as illustrated in 
Exhibit B-3. 
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Exhibit B-3 

Types of Space Used in Activities Observed, Spring 2005 



Types of space 


Percent of Observations 

(N=159) 


Classroom 


47 


Gym 


15 


Cafeteria 


15 


Auditorium 


13 


Outside playground 


4 


Other 


3 


Library 


1 


Hallway 


1 


Music room 


1 



Exhibit reads: 47 percent of activities observed in this study occurred in ciassrooms. 



Scaies Created from the Observation Instrument 

These scales reflect each of the five key youth development domains listed in the main 
report. However, indicators from the youth-directed relationship-building domain and the youth 
participation domain were combined into a single scale, strengthening the scale both 
conceptually and statistically. Thus, analysis of these observation data combined ratings on 
individual indicators into four scales, which are described below. Within each of these domains, 
the scales include specific indicators that were expected to be prevalent in most after-school 
activities, as well as some indicators that the study hypothesized would be harder for projects to 
implement and therefore more rarely observed. Like the individual indicators within the 
domains, each scale was computed on a scale of one to seven. 

Evaluators conducted ANOVA analyses to determine whether there were statistically 
significant differences in scale scores between different types of activities (e.g., homework help, 
academic enrichment, arts, sports). A Bonferroni post-hoc test was used to determine whether 
there were significant differences between pairs of activity types (for example, homework help 
and arts). Similarly, evaluators conducted analyses to determine whether there were statistically 
significant differences in scale scores for activities that targeted academic skills compared with 
those that did not. 
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Youth Relationship Building and Participation 



The youth relationship building and participation scale combines ratings from the following 
indicators: 

1 . All or most youth are friendly to each other. 

2. All or most youth show respect for one another. 

3. All or most youth show positive affect to staff. 

4. All or most youth are on-task. 

5. All or most youth listen actively and attentively to peers and staff. 

6. All or most youth have opportunities to make meaningful choices. 

7. All or most youth take leadership responsibility /roles. 



Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


.73 


4.49 


0.85 


2.29 


3.93 


5.00 


6.71 
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Staff- Youth Relationships 



The staff relationship building scale attempts to quantify the efforts of staff members in their 
interpersonal relationships with youth. The study created this scale by combining ratings from 
the following indicators: 

1. With all youth, staff use positive behavior management techniques. 

2. With all youth, staff are equitable and inclusive. 

3. Staff show positive affect toward youth. 

4. Staff attentively listen to and/or observe youth. 

5. Staff encourage youth to share their ideas, opinions and concerns. 

6. Staff engage personally with youth. 

Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.79 


4.58 


1.04 


2.00 


3.83 


5.33 


7.00 
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Skill Building and Mastery 



The Staff Skill Building and Mastery scale measures the efforts of after-school staff to promote 
learning and build proficiency in youth. Staff were evaluated based on the following items: 

1. Staff communicate goals, purposes, expectations. 

2. Staff verbally recognize youth’s efforts and accomplishments. 

3. Staff assist youth without taking control. 

4. Staff ask youth to expand upon their answers and ideas. 

5. Staff challenge youth to move beyond their current level of competency. 

6. Staff plan for/ask youth to work together. 

7. Staff employ two or more teaching strategies. 

Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.83 


3.77 


1.40 


1.00 


2.57 


5.00 


6.71 
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Activity Content and Structure 



The observation team used the items in the Activity Content and Structure scale to measure an 
activity’s overall quality. The scale includes the following items: 

1 . The activity is well organized. 

2. The activity involves the practice/a progression of skills. 

3. The activity challenges students intellectually, creatively, and/or physically. 

4. The activity requires analytic thinking. 

Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.88 


4.35 


1.69 


1.00 


3.00 


5.75 


7.00 
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Appendix C 

Out-of-School Time Program Observation Instrument 




Policy Studies Associates, Inc. 

1718 CONNECTICUT AVENUE, N W • SUITE 400 • WASHINGTON, D C 30009 

12021 939-9780 • 12021 939-5732 FAX • www poliCystuoics.com 



OUT OF SCHOOL TIME (OST) 
OBSERVATION INSTRUMENT 



2"'' Edition 
December 2005 



OUT OF SCHOOL TIME (OST) 
OBSERVATION INSTRUMENT 
Observer Procedures 



The Observation Instrument: An Overview 

The observation instrument provides site visitors at the out of school time program sites with a 
framework to capture and rate essential and observable indicators of positive youth development. 

The observation instrument includes the following: 

Cover Sheet: a checklist for capturing basic facts about the observed activity, such as 
activity type, staff roles, number of participants, and grouping patterns. 

Observation Notes: for recording observation notes within five youth-development 
domains, such as youth-directed relationships and participation and staff-directed 
relationships. 

Domain Item Ratings: a list of items under the five different domains. Observers rate each 
item on a scale of 1-7. 

Environmental Ratings: three questions regarding the context of each activity, such as the 
activity space and materials used. 

Observer’s Synthesis: for each domain, observers write 1-2 sentences describing the overall 
quality observed. 



Completing the Observation Instrument 

STEP 1: Each activity should be observed for 15 minutes. Site visitors begin their observations 
by orienting themselves to the activity setting, scanning the activity space against the items on the 
coversheet. However, only the introductory items (e.g., location, observer, date, time) should be 
completed at the onset of the observation. 

STEP 2: After sufficient orientation, the site visitor begins to take notes on what he or she 
observes within five broad quality domains. On the ‘Observation Notes’ page, site visitors describe the 
types of interactions, strategies, etc., that are observed for each domain. These descriptions should 
include SPECIFIC EXAMPEES of activities, QUOTES of youth and staff comments, and descriptions 
of the general affect in the setting. Comments should be brief, but in sufficient detail that they support 
the item ratings. Observers may use the back of the note sheet, if more recording space is needed. 
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While taking notes, observers should periodically scan each of the five domains and their 
indicators to verify that examples of all observable events are captured. In some observations, not all 
items within the domains will occur. 



After 15 minutes of observation , while still in the activity setting, complete Steps 3-5. 



STEP 3: Return to and complete the observation instrument’s coversheet. Then turn to the 
final page of the instrument and complete the three Environmental Context questions. 

STEP 4: Complete the “Observer’s Synthesis” section on the final page of the instrument. 

Write one or two sentences that describes or synthesizes the overall quality of each domain. 

STEP 5: Finish the observation by giving each domain item a rating of 1-7. The ratings are: 

• 1 = Exemplar is not evident 

• 3 = Exemplar is rarely evident 

• 5 = Exemplar is moderately evident, or implicit 

• 7 = Exemplar is highly evident and consistent 

To select a rating, first move to the ODD NUMBER that most closely reflects how evident and 
pervasive an item is. If that number does not precisely reflect the level of evidence observed, then move 
down or up to the adjacent even number that more accurately reflects the item’s level of presence within 
an activity. 

Neutral Items: The “5” rating is also used in cases where the exemplar is implicit in the 
activity. Eor instance, if youth are generally friendly to each other throughout the observation, but most 
do not go beyond the casual, friendly interaction, the rating would be a “5”. 

What if I am observing more than one staff in an activity? If there is more than one staff 
working in the room, ratings involving staff should be balanced across staff interactions. Eor instance, if 
one staff member uses effective behavior management techniques while the other resorts to threats and 
put-downs, then the rating would reflect an average of the two approaches (e.g., a rating of ‘3’ or ‘4’). 



HELPFUL HINT: When you are rating an item, first read its definition and underiine those things that 
you were able to observe. This will help highlight how much of an item’s full definition was captured. 
Then determine the prevalence of the definition (or portion thereof). 
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Sampling 



How to organize your observations on site. Site visitors should review the program’s activity 
schedule during prior to the program start on the day of their visit. Site visitors should select activities 
to be observed from the project schedule, being mindful of the following: 

1. Ensure that the broadest scope of activities is observed by selecting activities across five 
activity types: (1) Homework Help/Tutoring/Test Prep, (2) Academic Enrichment, 

(3) Arts Activities, (4) Eitness, and (5) Other Enrichment (e.g.. Girls Group). A sample 
observation matrix is below. Please note that while snack can be an important part of an 
after-school project, observations should be geared toward more substantive activities. 

2. Observe across grade levels as well as activities. 

3. Verify with the program coordinator that each scheduled activity will be occurring in the 
location stated. 

4. Verify the staff who will be present in each activity, noting level of education or special 
skills (e.g., high school student, certified teacher, arts specialist). 

How many observations should site visitors conduct? If you are testing for inter-rater 
reliability, for each visit, three observations should be co-conducted. The two site visitors should 
observe these three activities and complete the observation instrument as described above. Be sure to 
check “Yes” in the co-observed checkbox and enter both observers’ initials. 

If you are co-observing with another site visitor, once you are out of the activity space, take 5-10 
minutes to compare your ratings. The purpose of this comparison exercise is to get to a more mutual 
understanding of the definitions, ratings and quality of activities and interactions observed. DO NOT 
CHANGE YOUR RATINGS, even if you ultimately agree with how your co-observer has rated an item. 
Also, you do not need to reconcile your ratings on another copy of the form. 

In addition to the three co-conducted observations, each site visitor should observe eight to ten 
after-school activities over two afternoons (or approximately 5-6 hours). This is a total of 1 1 to 13 
observations per site visitor, which is the equivalent to six or seven observations each per day, or just 
under two hours of observing per 2.5 - 3 hour after-school session. 

Eollowing the above plan, each site visit team should return 22 to 26 completed observation 
instruments at the end of each two-day visit. 
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Activity Observation Matrix 



If a two-person site visit team is conducting a two-day visit, this is an example of how visitors can 
organize their observations. The matrix below serves as a guide for selecting activities. 

During the two-day site visit, each visitor should conduct a total of 1 1 to 13 observations, 
including 3 co-observations and 8-10 individual observations. Each visitor should aim to visit one 
activity in each of the 13 cells in order to observe a range of activities and of grade levels. 



Activity Type 


Younger Group 

(grades K-2 or 6/7) 


Older Group 

(grades 3-5 or 7/8) 


Co-Observation 


Homework help, 
tutoring, test 
preparation 






1. 


Content-based 
academic enrichment 

(e.g. literacy, science, 
newspaper) 






Arts: Visual; Performing 
(including dance) 






2. 


Fitness: Open 
Sports/Movement; 
Structured Athletics 
(including Karate) 






Other Enrichment: (e.g. 
Outing Club, Girls’ Group, 
Venture Club, Investment 
Club, College Club) 






3. 



Additional Instructions: 

• If no content-based academic enrichment activities are offered, visit additional homework 
help/test preparation activities. If no “other enrichment” activities are offered, visit a mix of 
additional fitness and arts activities. 

• Visitors should co-observe either a homework help or content-based academic enrichment 
activity and either an arts or fitness activity, as well as an “other enrichment” activity, and 
should plan to see activities that include both the older and younger students in the program. 

• If activities are limited, site visitors should still observe the required 6-7 activity segments per 
day, observing an activity more than once, if necessary. 

• Observe all scheduled activities planned for that day, regardless of the quality . If a planned 
activity has been replaced or is not occurring, note this on the observation recording form and 
move on to the next selected activity. You will need to replace this activity with another one 
from the day’s schedule. 
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OUT OF SCHOOL TIME (OST) OBSERVATION INSTRUMENT 
DEFINITIONS FOR OBSERVATION CATEGORIES 




OST OBSERVATION DEFINITIONS 



ACTIVITY NAME 


The title of the activity, as given by the project 


ACTIVITY OVERVIEW 


1 -2 sentence description of activity 


ACTIVITY TYPE (check all that apply) 


Homework help and/or 
test preparation 


Check this category when youth work on homework or specially assigned skill drill. If youth participate in 
academic activities that are not homework, mark “Academic activities.’’ If youth are working on an in-depth, 
expanded academic assignment, but it IS homework, MARK THIS category. 


Tutoring 


Tutoring refers to skill-focused academic assistance to individuals or small groups with common learning needs. 
It is distinguished from homework or other activities in that it aims to strengthen or remediate specific academic 
skills. 


Academic activities (not 
homework) 


This item refers to an activity that develops broad-based conceptual/cognitive learning; it may or may not be 
directly related to school content, but it includes enriched supplementary instructional content that goes beyond 
homework, tutoring, and rote skill practice. Examples could include purchased literacy or numeracy curriculum, 
research, science, social studies, newspaper club, poetry club, or school store. If supplementary activities are 
offered but are not in-depth, or cognitively rich, do not mark this activity type. 


Story reading/listening 


Reading or listening to a story. 


Visual and Textile Arts 


Specialized visual arts-based activities such as painting, drawing, clay/sculpture, photography, knitting, sewing, 
etc. that include structured lessons in appropriate use of techniques, materials, and design elements to create 
products. Often the activity involves creating projects in multiple stages that may take several days, weeks or 
months to complete. 


Dance 


Dance lessons, practices, or performances. This area does NOT include athletic types of movement such as 
cheerleading or martial arts, which are marked in one of the “Sports” categories. 


Music 


Instrumental or vocal lessons, practices, or performances. 


Drama 


Drama lessons, practices, or performances. 


Crafts 


Non-specialized arts activities, using crayons, paints, yarn, etc. Involves creating a product for fun, but does not 
involve basic instruction in design or technique. Often is a short-term project, involving a product but requiring 
only the activity period or a few days to complete. 


Sports — practicing or 
learning a skill 


Preparation and training in a sport or athletic skill of any kind, including basketball or baseball skill clinics, martial 
arts, gymnastics, weight lifting, yoga, cheerleading practice. 


Sports — competitive or 
non-competitive 
physical game 


Supervised or non-supervised games using athletic skill, indoors or outdoors, such as basketball or baseball 
games. 


Open, unstructured time 
(e.g., table games, 
internet, free play) 


This category refers to focused activity, freely chosen by youth and not structured (but may be supervised) by 
adults; typically, youth have invented, selected, or identified a task and are implementing it without adult 
direction. 


Community service 


Planning or assisting with projects that support the quality of community life or foster program-school or 
community-school linkages. 


College/career 

planning/preparation 


Activities directly involved in career or college planning or preparation. 


Cultural awareness 
clubs/projects 


Activities/projects that develop cultural, religious, or ethnic awareness, understanding, or identity. 


Conflict resolution 


Activity specifically designed to impart conflict resolution skills, such as positive communication, tolerance (e.g. 
ethnic, religious), peer mediation, diffusing anger, etc. 


Other 


Describe here activities observed that cannot be categorized or grouped above, e.g. chess club or student 
government. 
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TYPE OF SPACE 

(Check only one) 


Classroom 


Check the one location in this category that best applies to the setting in 
which the activity takes place. If “other” is marked, specify the type of 
space used. 


Gym 


Computer lab 


Library Outside 


Cafeteria 


Auditorium 


Music room 


Outside playground 


Hallway 


Art room 


Other 


TOTAL STAFF 

Count and record in 
the space on the right 
the NUMBER of staff 
in each activity. The 
number should 
correspond to staff 
levels/skills. Do not 
count middle school 
students. 


High School Student 


“High school students” are in grades 9-12. 


College Student or 
Young Adult 


“College students or young adult” refers to staff who have finished or left 
high school and are approximately 18-24 years of age. 


Certified Teacher 


“Certified teachers” are classroom teachers from the host school OR another 
school. 


Specialist/Other 

Professional 


A “Specialist” has skills or talents which are the focus of the activity, such 
as a musician, artist or chess instructor. “Other professional” include social 
workers, guidance counselors, or other staff with professional degrees. 


Other Adult 


“Other adult” refers to staff approximately 25 years of age or older; these 
staff are school aides, paraprofessionals, parents, community members, etc. 
who may or may not have an undergraduate degree. 


TOTAL 

PARTICIPANTS 

Count and record the 
number of girls and the 
number of boys in the 
activity. 


Total Number of Girls 


If in doubt about gender, make an educated guess rather than disrupting the 
class. 


Total Number of Boys 


GRADE LEVELS 

(Circle all that apply) 


Grade Level Observed 


Circle the all grades that are represented in the classroom or activity space. 
More than one category can be circled, if appropriate. 


PARTICIPATION 
TYPE (Check one) 


By age or grade 


Indicate the predominant way that youth are selected to participate in this 
activity. 


By interest (child’s 
choice) 


All attendees (in the 
project) 
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Skill building 


Check “Skill-building”, if youth are engaged in an activity that builds upon 
a previously learned skill and is intended to help youth reach the next level 
of mastery, or if a new skill is learned and built upon. 

An example may be a new piece of dance choreography, rehearsing the next 
scene in a play, revising a piece of writing, or continuous practice and 
improvement of a recital piece. 


SKILLS SECTION 

(Check one) 


Skill practice or 
reinforcement 


Check “Skill Practice”, if youth are using or reinforcing a skill already 
learned but the activity is not intended to help youth reach the next level of 
mastery. An example might include completing a math skills sheet. 




Neither 


Check “Neither”, if activities do not build or reinforce skills already learned. 




This is a homework 
activity. 


Check “This is a homework activity”, if youth are working on homework. 




Physical/athletic 


Athletics, games, skills of physical sport (including martial arts, yoga, step, 
cheerleading, gymnastics, etc.). 


PRIMARY SKILL 
TARGETED 


Artistic 


Artistic skills, working in any medium (visual, musical, dance, dramatic, 
photographic, video, etc.). 


Math/numeracy 


Mathematics learning, skill development, practice. 


Only complete this area if 


Reading/literacy/ 

writing 


Reading/language arts learning, skill development, practice. 


SKILL BUILDING or 
SKILL PRACTICE were 


Decision making/ 
problem solving 


Developing skills in making practical or conceptual decisions or solving 
practical or conceptual problems. 


checked in the Skills 
Section above. 


Interpersonal 

communication 


Developing skills that involve self or group reflection, negotiating, 
interaction, and/or improvement of connections/relationships among people. 




Other 


Check this category AND DESCRIBE here specific other content areas 
(e.g., science, social studies, foreign language) or skills developed that are 
not listed above. 
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ENVIRONMENTAL CONTEXT QUESTIONS 


1. Is the level of adult supervision appropriate to 
activity and age group? 


Mark “Yes”, if the number of adults in the room allows for safety, 
activity implementation, and individualized attention to youth. 

Mark “No”, if there are too few adults to ensure participant safety, to 
implement the activity, or to provide adequate support to individual 
participants; also mark “No” if there are too many adults, which is 
deterring youth interactions and/or leadership. Provide an explanation of 
the “no” response in the space provided. 


2. Is the workspace conducive to the activity? 


Mark “Yes”, if the physical work space is conducive to the group size 
and activity type. 

Mark “No”, if the physical work space has evident hazards, is not 
conducive to the group size, or is inappropriate for the activity type. 
Provide an explanation of the “no” response in the space provided. 


3. Are necessary materials available and in 
sufficient supply? 


Mark “Yes”, if participants have an adequate supply of the 
materials/tools they need to make progress on tasks or activities, and if 
the quality or condition of the materials is adequate (materials/equipment 
are in working order, i.e., they are not piecemeal or broken) and 
activities/tasks can be completed with what is available. 

Mark “No”, if materials are in poor working condition, or if there is an 
inadequate supply, to the extent that the activity is not effective for 
youth. Provide an explanation of the “no” response in the space provided, 
such as “Not enough instruments for all youth.” 
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OUT OF SCHOOL TIME (OST) 
OBSERVATION INSTRUMENT 

COVERSHEET 



CO-OBSERVED? 

□ Yes □ No 

CO-OBSERVERS ’ INITIALS : 
I. 2. 



Program 


Observer 


Observation 


Room Number: 


Date 


Start Time: 


End Time: 


ID/Location: 


Initials: 


Number: 




(MM/DD/YR); 







ACTIVITY NAME: 



ACTIVITY OVERVIEW 



(1-2 sentence description): 



ACTIVITY TYPE 


V 

ata* 


TYPE OF SPACE 


one 


TOTAL 

PARTICIPANTS 


# 


Homework Help/Test Prep 




Classroom 




Total Number of Girls 




Tutoring 




Gym 




Total Number of Boys 




Academic activities (not 
homework) 




Computer Lab 




GRADE LEVELS 

(circle all that apply) 


Story reading/listening 




Library 




K 


1 


2 


3 


4 


5 


6 


Visual arts 




Cafeteria 




7 


8 


9 


10 


11 


12 


other 


Dance 




Auditorium 




PARTICIPATION TYPE 


V 

one 


Music 




Art Room 




By age or grade 




Drama 




Music Room 




By interest (child’s choice) 




Crafts 




Hallway 




All attendees (in the project) 




Sports — practicing/learning a skill 




Outside Playground 








Sports — playing competitive or 
non-competitive physical games 




Other: 




*ata = all that apply 


Open, unstructured time (e.g., 
table games, internet, free play) 




TOTAL STAFF 


# 




Staff-assigned learning games 
(dominos, chess, etc.) 




High School Student 




Community service 




College Student or Young Adult 




College/career preparation 




Certified Teacher 




Cultural awareness clubs/projects 




Specialist or Other professional 




Other: 




Other Adult 





At the end of the observation, please indieate what type of skill 
development, if any, took, plaee in this aetivity. 


PRIMARY SKILL TARGETED IN SKILL-BUILDING 


V 

one 


SKILL DEVELOPMENT 


V 


Physical/ Athletic 




one 


Artistic 




Skill-building 


( oni|i’:_U- skills 


Math/N umeracy 




area 


Reading/W riting/Literacy 




Skill practice/reinforcement 


C’omplete skills 
area 


Decision-making/ Problem-solving 




Neither 


Dll nut iriimpk’::' 
skills area 


Interpersonal Communication 




This is a homework activity 


Do not complete 
skills area 


Other: 
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OST OBSERVATION INSTRUMENT DOMAIN ITEM RATINGS 



The following are five youth development domains that include four to seven indicators of youth development 
practices within each. The purpose of the observation instrument is to measure the extent to which these indicators - also 
called “items” - may or may not be present in each 15-minute observation segment, necessarily negative. 

After 15 minutes of observation, assign a rating of 1 (not evident) to 7 (highly evident and consistent) for each item below. 
To select a rating, first move to the ODD NUMBER that most closely reflects how evident and pervasive an item is. If that number 
does not precisely reflect the level of evidence observed, then move down or up to the adjacent even number that more accurately 
reflects the item’s level of presence within an activity. Note that each item/indicator may not be present/applicable in each 
observation; therefore, a rating of “1” is not necessarily negative. 

The “5” rating is also used in cases where the exemplar’s presence is implicit within the activity. For instance, if 
youth are generally friendly to each other throughout the observation, but most do not go beyond the casual, friendly 
interaction, the rating would be a “5”. 

RATINGS: 



Exemplar is not Exemplar is 

evident rarely evident 









Exemplar is 
moderately 
evident, or 
implicit 



Exemplar is 
highly evident 
and consistent 



RELATIONSHIP BUILDING : all or most 
YOUTH 


PARTICIPATION; all or most YOUTH 


A 


Are friendly and relaxed with one another. Youth 
socialize informally. They are relaxed in their 
interactions with each other. They appear to enjoy one 
another’ s company. 




Are on-task. Youth are focused, attentive, and not 
easily distracted from the task/project. They follow 
along with the staff and/or follow directions to carry-on 
an individual or group task. 


b 


Respect one another. Youth refrain from causing 
disruptions that interfere with others accomplishing their 
own tasks. When working together, they consider one 
another’s viewpoints. They refrain from derogatory 
comments or actions about the individual person and the 
work s/he is doing; if disagreements occur, they are 
handled constructively. 


G 


Listen actively and attentively to peers and 

staff. Youth listen and respond to each other and staff. 
They appear interested in what others have to say. 

They look at peers and/or staff when they speak, and 
they provide concrete and constructive feedback about 
ideas or actions. 


C 


Show positive affect to staff. Youth interact with the 
staff, and these interactions are generally friendly 
interactions. For example, they may smile at staff, laugh 
with them, and/or share good-natured jokes. 


H 


Contribute opinions, Ideas and/or concerns to 
discussions. Youth discuss/express their ideas and 
respond to staff questions and/or spontaneously share 
connections they’ve made. This item goes beyond 
basic Q&A and refers to sharing that is part of the 
activity and within the class norms. Calling out - or 
disruptively talking out of turn - is not part of this 
item. 


D 


Assist one another. One or more youth formally or 
informally reach out to help/mentor peers and help them 
think about and figure out how to complete a task. This 
item refers to assistance that is intentional and prolonged, 
going beyond answering an incidental question. 


I 


Have opportunities to make meaninaful 
choices. Within this activitv, vouth choose what thev 
do, how they do it, and/or with whom they collaborate, 
and they experience the consequences of their choices. 
This item refers to genuine options, not simple choices 
such as choosing between two types of games, or two 
sets of homework pages. 


L 


Are collaborative. Youth work together/share 
materials to accomplish tasks. This item is different from 
item D (above), as in collaboration, youth are equal 
partners in the work (rather than one student 
assisting/mentoring/tutoring another). This item can 
include working together on assigned teams, if youth are 
working together to get a better result. 




Take leadership responsiblllty/roles. Youth have 
meaningful responsibility for directing, mentoring or 
assisting one another to achieve an outcome; they lead 
some part of the activity by organizing a task or a 
whole activity, or by leading a group of youth within 

the activity. . 
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RELATIONSHIP BUILDING: with all youth, STAFF... 


INSTRUCTIONAL STRATEGIES: STAFF... 


K 


Use positive behavior management 
techniques that allow for youth to accomplish the 
activity’s objectives. They set consistent limits and 
communicate clear expectations for behavioral 
standards, and these are appropriate to the age of the 
youth and the activity type. When disciplining youth, 
they do so in a firm manner, without unnecessary 
accusations, threats, or anger. 


R 


Communicate goals, purpose, expectations. Staff make 
clear the value and purpose of what youth are doing and/or 
what they expect them to accomplish. This item goes beyond 
how youth are expected to behave (which would be captured in 
item K). 


L 


Are equitabie and inciusive. Youth are provided 
equal opportunity to participate in an activity and are 
rewarded/disciplined similarly for like actions. Staff 
encourage the participation of all youth, regardless of 
gender, race, language ability, or other evident 
differences among students. They try to engage 
students who appear isolated; they do not appear to 
favor a particular student or small cluster of students. 


S 


Verballv recoanize vouth’s efforts and 
accomplishments. Staff acknowledge participation and 
progress in order to encourage youth. 


M 


Show positive affect toward youth. Staff 
interact with youth, and these interactions are 
generally friendly. For example, their tone is caring, 
and/or they use positive language, smile, laugh, or 
share good-natured jokes. 


T 


Assist youth without taking control. Staff may coach, 
demonstrate, or employ scaffolding techniques that help youth 
to gain a better understanding of a concept or complete an 
action on their own. Staff refrain from taking over a task or 
doing something on behalf of the youth. This assistance goes 
beyond checking that work is completed. 


\ 


Attentively listen to and/or observe youth. 

Staff look at youth when they speak and acknowledge 
what youth have said by responding and/or reacting. 
They pay attention to youth as they complete a task 
and appear interested in what they are saying/doing. 


r 


Ask youth to expand upon their answers and ideas. 

Staff encourage youth to explain their answers, evidence, or 
conclusions. They may ask youth ‘why’, ‘how’ and ‘if 
questions to get them to expand, explore, better clarify, 
articulate, or concretize their thoughts/ideas. This item goes 
beyond staff-elicited Q&A. 


0 


Encourage youth to share their ideas, 
opinions and concerns. Staff activelv elicit vouth 
ideas, opinions and concerns through discussion 
and/or writing. This item goes beyond basic Q&A. 




Challenge youth to move beyond their current level 
of competency. Staff give constructive feedback that is 
meant to help youth to gauge their progress. Staff help youth 
determine ways to push themselves intellectually, creatively, 
and/or physically. 


P 


Engage personally with youth. Staff show 
interest in youth as individuals, ask about youth’s 
interests, and engage about events in their lives. 


V‘,' 


Employ varied teaching strategies. In order to engage 
students and/or reach those with different learning styles, staff 
diversify instructional strategies, which may include the use of 
two or more of the following: direct instruction, coaching, 
modeling, demonstrating, or others. Varied instructional 
strategies can occur simultaneouslv and/or seauentiallv within 
the observation period. This item does not include coupling a 
staff-directed instruction with youth working together, as 
described above. 


0 


Guide for positive peer interactions. Staff 
intentionally encourage positive interactions and/or 
directly teach interpersonal skills. They teach these 
skills through planned activity content or through 
intervening constructively and calmly to address 
bullying or teasing behavior, redirecting youth and/or 
explaining or discussing why negative behavior is 
unacceptable. This item does not refer to behavior 
management, as described above (see item K). 


''' 


Plan for/ask youth to work together. Staff plan for 
and/or ask youth to work together, solve problems, and/or 
accomplish tasks. The focus of the activity is youth to youth, 
rather than youth to staff This item goes beyond staff assigned 
teams for competitive games and sports. In the case of staff 
assigned teams, staff would also need to be directing youth to 
collaborate, plan, devise, etc., in order for this item to be rated 
as staff asking youth to work together. 
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CONTENT AND STRUCTURE: ACTIVITY 


Y 


Is well organized. Activity has clear 
goals/objectives; there is evidence of a clear lesson 
plan and process(es), and tasks can be conducted in 
the timeframe available. If special materials are 
needed, they are prepared and available. 


AA 


Involves the practIce/a progression of skills. Activity 
involves the progressive development, learning OR practicing 
of skills needed to complete tasks or to participate. 


Z 


Challenges students Intellectually, creatively, 
developmentally, and/or physically. Activity’s 
level of challenge is not so difficult that youth have 
trouble participating successfully and not so easy that 
youth master skills quickly and become bored. 


BB 


Requires analytic thinking. Activity calls on students to 
think about and solve meanineful problems and/or iuasle 
multiple activities or dimensions to accomplish a task. For 
example, the activity requires youth to hold two or more ideas 
constant at the same idea, and/or understand and apply 
sequencing or patterns. 



ENVIRONMENTAL CONTEXT 






1. Is the level of adult supervision appropriate to activity and age group? 


Yes 


No 


If no: ->Why not? 






2. Is the work space conducive to the activity type? 


Yes 


No 


If no: ^Why not? 






3. Are necessary materials available and in sufficient supply? 


Yes 


No 


If no: ->Why not? 







OBSERVER’S SYNTHESIS 

Before leaving the activity setting, please provide a 1-2 sentence description of the overall quality of each domain within this 
activity. 


DOMAIN 


OBSERVER’S DESCRIPTION 


Youth-directed relationships: youth are supportive 
and respectful of one another and staff. 




Youth participation: youth exhibit engagement in the 
activity; there are opportunities for their input and 
leadership. They appear to enjoy the activity content. 




Staff-directed relationships: adults provide guidance 
and emotional support; they take interest in the youth and 
their ideas. 




Instructional strategies: staff strategies are geared 
towards encouraging youth to push beyond their present 
level of competency. 




Activity content and structure: activities are planned 
and well organized; challenge level is appropriate to age; 
there are opportunities for problem solving. 
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OBSERVER’S NOTES 



Observer’s Initials: 


Observation #: 


Activity Name: 


RELATIONSHIP BUILDING: all or most YOUTH 


PARTICIPATION: all or most YOUTH 






RELATIONSHIP BUILDING: with all youth, STAFE 


INSTRUCTIONAL STRATEGIES: STAFF 






ACTIVITY CONTENT AND STRUCTURE 





c-i 



RELATIONSHIP BUILDING: aU or most YOUTH 


PARTICIPATION: all or most YOUTH 






RELATIONSHIP BUILDING: with all vouth, STALE 


INSTRUCTIONAL STRATEGIES: STALE 






ACTIVITY CONTENT AND STRUCTURE 
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Appendix D 

Profiles of High-Performing Projects 




CAMBA KIDS AFTER-SCHOOL PROJECT 
Bedford-Stuyvesant, Brooklyn 



Project Overview 

Project Start Date: 2000 
Grades Served: K-8 
2004-05 Budget: $208,000 
Enrollment: 165 
Attendance Rate: 75% 



Population Served 

Eligible for Eree/Reduced Price Eunch: 99% 
Recent Immigrants: 5% 

English Eanguage Eeamers : 1 % 

Eligible for Special Education Services: 8% 



According to site coordinator Charmaine Noel, the project’s goal is to expose participants 
to a diverse program of academic, artistic, recreational, and developmental learning 
opportunities. Academic activities include homework help and activities selected from 
Developmental Studies Center’s KidzLit and KidzMath and Scholastic Publishers’ Math and 
Reading Laboratories. Theme-based and project-based learning and selected curricula on 
conflict resolution teach students to take responsibility for their own actions and to assume 
leadership. Older students participate in YouthEink, a dialogue guided by a social worker about 
navigating risks such as gang affiliation and drug use. Activities available on Eriday Club Days 
and Eun Wednesdays are based on participant surveys. In a typical week, CAMBA Kids 
activities include: homework; KidzLit and KidzMath-, science projects; library time; dance, drums 
and recorders, or band practice; basketball or other sports; art activities such as self-portraits, 
collages, and masks; conflict resolution or peer dialogue on risk avoidance; cooperative games; 
and indoor or outside open play. 

Staff are mainly youth development paraprofessionals and young adults, although two 
group leaders are certified teachers. Several staff also work as aides in the partner school. 
Assistant leaders are college students, AmeriCorps staff, and adults from the community, some 
of whom have had children in the project. The arts program is staffed by a professional dancer 
and a musician. Ms. Noel assigns group leaders and assistants according to their grade 
preferences, skills, and interests, headers remain with their groups ah year long, but artists rotate 
so that each group has at least one arts activity each week. The librarian and the mathematics 
coach from the host school work part-time in the after-school project to shape academic 
enrichment activities so that they are content-based and coordinated with school-day curricula. 

An essential part of the project’s success is its strong communication with the day-school 
staff, including the principal, teachers, custodians, cafeteria staff, and security guards. Ms. Noel 
and the principal meet bi-weekly and more often if necessary, and she serves on the school’s 
leadership team. 
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COMMITTEE FOR HISPANIC CHILDREN AND FAMILIES 

South Bronx 



Project Overview 

Project Start Date: 1999 
Grades Served: K-8 
2004-05 Budget: $285,000 
Enrollment: 244 
Attendance Rate: 78% 



Population Served 

Eligible for Eree/Reduced Price Eunch: 99% 
Recent Immigrants: 4% 

English Eanguage Eeamers: 36% 

Eligible for Special Education Services: 7% 



The after-school schedule in this project changes approximately four times a year, as 
students’ school schedules and needs change. In winter, the focus is on teaching students 
academic skills in preparation for state and city achievement tests. “We ensure that they are 
reading and that they are studying for the test,” said site coordinator Helena Yordan. 

A project-wide theme unites activities from January to June. Groups of participants 
select aspects of the theme to research on-line and in the library. In May and June, they engage 
in outside activities and field trips as well. Participants then integrate their research into the 
project’s culminating performance. This might involve choreographing a dance, writing and 
acting in a play, or creating props and scenery in visual arts activities. 

The project is highly structured and participants have little down-time; every activity 
supports students’ academic or social development. Three times a year, the site coordinator asks 
regular school-day teachers to report on the academic progress of each project participant. An 
educational specialist, who is also the school’s science developer, helps align after-school 
activities with the school day curriculum, identifies instructional materials, provides professional 
development, and supports group leaders in other ways as needed. 

At the start of each project year, staff participate in 15 hours of required training that 
helps them understand the academic standards and content in the grade level at which they will 
be working. Staff also receive an annual project planning calendar that includes the project’s 
activity goals and schedule, as well as the school calendar and instructional goals. Within these 
parameters, staff select activities for their groups. 

Group leaders submit lesson plans and keep logs of completed activities. They are 
expected to have their activity plans up-to-date, along with back-up activities should a specialist 
be absent or a scheduled activity end sooner than expected. Each group leader keeps his or her 
lesson plans and materials in a box so that substitute leaders have easy access to them. Staff also 
attend weekly (paid) staff development meetings to review the week’s activities and to address 
programmatic topics and the needs of particular students. 
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CYPRESS HILLS LOCAL DEVELOPMENT CORPORATION 

KIDS CLUBHOUSE 
Cypress Hills, Brooklyn 



Project Overview: 

Project Start Date: 1999 
Grades Served: K-4 
2004-05 Budget: $285,000 
Enrollment: 289 
Attendance Rate : 8 1 % 



Population Served 

Eligible for Eree/Reduced Price Eunch: 97% 
Recent Immigrants: 7% 

English Eanguage Eeamers: 28% 

Eligible for Special Education Services: 1% 



The goal of Kids Club is to strengthen participants’ academic skills and develop a strong 
sense of social and civic responsibility among participants. Each afternoon from 3:00 p.m. until 
5:45 p.m., participants engage in 45 minutes of supervised homework, followed by two 
enrichment activities, which are coordinated with supplementary academic activities offered by 
the host school. 



KidzLit and KidzMath curricula, hands-on science and social studies projects, and 
supervised homework help build reading, math, and thinking skills through activities such as 
read-alouds and dramatizations of books and, for math and science, through counting games, 
experiments, and investigations. Project-based activities and active learning games, such as 
Jeopardy, academic trivia, and geography bees, are infused into academic enrichment. Other 
ventures involve the production of an after-school project newspaper, self-governing activities, 
and field trips to local museums, cultural festivals around the city, and foreign embassy offices. 
On these visits, participants decide on questions to ask, and everyone keeps a journal about their 
experiences. Groups may also study a country by visiting a restaurant to sample its food, 
cooking native recipes, and creating a recipe book for their parents. Under the direction of 
AmeriCorps staff, some groups become involved in service learning and fund-raising activities. 

The project encourages a family-like structure. Participant groups remain together for a 
full year. This gives leaders a chance to get to know parents and to develop strong relationships 
with participants’ day-school teachers. Site coordinator Eee Arroyo also builds staff 
relationships by organizing informal social events such as pizza parties and bowling outings. 
When staff are out because of illness or family needs, she writes a note making clear the 
individual is missed. If staff are not pulling their weight in the project, she addresses the 
problem firmly but positively. 

The after-school project maintains a solid relationship with the principal and faculty of its 
host school. Ms. Arroyo reported that she speaks with the principal every day. Shared school 
and project staff also add continuity and communication. One group leader is the school’s parent 
coordinator; another school-day paraprofessional helps maintain Kids Club participants’ 
academic records. 
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EAST SIDE HOUSE AFTER-SCHOOL PROJECT 
Mott-Haven, Bronx 

Project Overview Population Served^ 

Project Start Date: 1999 Eligible for Free-Reduced Price Lunch: 99% 

Grades Served: K-8 Recent Immigrants: 7% 

2004-05 Budget: $290,000 English Language Learners: 23% 

Enrollment: n/a Eligible for Special Education Services: 7% 

Attendance Rate: n/a 

Meeting from the end of school at approximately 3:00 to 5:45 p.m., participants engage 
in three 45-minute activities each day. Academic enrichment includes homework help and 
exposure to content-based exploratory curricula, such as KidzLit, Voyager, and Junior 
Achievement programs. Participants also choose from a menu of arts and recreational activities, 
including dance, arts and crafts, structured recreational games and free play, theater, mask- 
making, ceramics, choir practice, movement/dance, and instrumental music. Kindergarten 
through eighth-grade participants work in grade-level groups led by an activity leader and an 
assistant. Occasionally, arts activities bring several grades together or older participants assist 
younger ones with homework and recreational pursuits. 

“For a lot of kids, this is their first dance class or art class,” noted site coordinator 
Pleshette McKnight. Some students “breathe through drawing or [do very well] creating a 
sculpture. When you’re learning academically, there’s an artistic side such as planning, 
decision-making, and follow-through. The two worlds are very parallel.” 

Other project priorities are conflict resolution and community -building through cultural 
understanding. The project has adopted two different conflict resolution curricula, one sponsored 
by Global Kids and the other by Educators for Social Responsibility (ESR). Both these 
organizations train staff in confronting and reducing tensions among cultures and groups. The ESR 
curriculum. Resolving Conflict Peacefully, draws on scenarios in which students role-play conflicts 
and solutions, and talk about traditions in their native countries. 

Leaders come largely from the nearby community, and most speak the children’s native 
languages. East Side Settlement House also offers social services to participants’ families, 
providing them with extra guidance, job counseling, or access to other resources they or their 
children may need. Staff receive regular training in how to identify and recommend help for 
neglected or abused students. Together, the training and the social services partnership with East 
Side increases the likelihood that families who need these services have access to them. 



* This project moved from one Bronx school to another after the student data were collected and analyzed but 
before this study’s visit to the program. The population data shown here characterized the program participants who 
achieved the learning gains. The narrative description is drawn from the site visit. 
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MASPETH TOWN HALL AFTER-SCHOOL PROJECT 

Woodside, Queens 



Project Overview 

Project Start Date: 2000 
Grades Served: K-6 
2004-05 Budget: $450,000 
Enrollment: 431 
Attendance Rate: 73% 



Population Served 

Eligible for Eree/Reduced Price Eunch: 73% 
Recent Immigrants : 1 1 % 

English Eanguage Eeamers: 25% 

Eligible for Special Education Services: 9% 



Maspeth Town Hall partners with its host school to offer academics, recreation, and arts 
activities in a safe, healthy after-school environment that gives participants opportunities to grow 
and thrive. Site coordinator Steven Powers aims to give participants a broad exposure to a range 
of activities. The formal academic enrichment period consists of 90 minutes of structured and 
supervised activities and 45 minutes of unstructured, informal interaction. In a typical week, 
participants select from homework, computers, literacy and math enrichment, yearbook 
development, drama, arts and crafts, board games, soccer, playground activities. Junior 
Achievement, chorus, and clubs. 

Staff encourage young leaders to take ownership of their ideas by organizing groups with 
common interests. Girls who wanted a dance program, for example, worked with the site 
coordinator to develop that activity. Other participants worked with staff from AmeriCorps to 
organize blood drives and other service projects. 

An assistant principal who works during the day in a nearby high school supervises 
academic activities and coordinates the remediation program with the school. Certified teachers 
help youngsters who need academic assistance. A lower staff-to-student ratio than in the project 
overall gives special education participants extra support. These participants are also eligible for 
supervised transportation home at the end of the day. 

School-day teachers and academic coaches provide much of the on-site staff training. In 
addition, special education teachers regularly talk with staff about inclusion practices. At other 
times, the project may draw upon expertise in its community, as it did when local nurses trained 
staff in adolescent health issues. 



Most often, current staff recommend new recruits for open staff positions, a system that 
builds strong bonds between established and new staff and contributes to an atmosphere of 
friendliness. The project also hires staff from area high schools. A number of these staff have 
gone on to study education. 
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RALPH-LINCOLN SERVICE CENTER’S 
MALCOLM X ACADEMY 
Crown Heights, Brooklyn 



Project Overview 

Project Start Date: 2000 
Grades Served: K-8 
2004-05 Budget: $225,400 
Enrollment: 229 
Attendance Rate: 46% 



Population Served 

Eligible for Eree/Reduced Price Eunch: 100% 
Recent Immigrants: 3% 

English Eanguage Eeamers: 0% 

Eligible for Special Education Services: 5% 



The Malcolm X Academy, operated by Brooklyn’s Ralph-Eincoln Service Center, works 
to create an after-school environment so compelling that participants will attend school in order 
to go to the project after school. Erom 3:00 to 6:00 p.m. each school day, the Academy offers, in 
addition to homework help and tutoring, dance, arts and crafts, drama, theater, and other 
enrichment opportunities. The safe environment and a healthy meal are designed to keep 
participants away from unsupervised streets, says site coordinator Barton Adams. 

Participants hone their talents in activities that interest them and show them off in year- 
end performances and exhibitions. These culminating performances and exhibitions, attended by 
parents and community members, develop a sense of accomplishment that participants rarely 
experience during the school day, according to Mr. Adams. 

The project offers Young Men’s and Young Women’s programs to middle-grades youth. 
These programs give youth the opportunity to talk informally about personal issues that are on 
their minds. Youth also participate in physical activities, such as yoga for girls and two-hand 
touch football for boys. With this program, Mr. Adams believes he has found activities that 
middle school youth like and will commit to attending. 

By taking advantage of programs in the community, the project offers a wider range of 
activities than it can provide on its own. Some participants recently attended an architectural 
workshop offered by the Brooklyn Center for Urban Environment in which they documented 
design features in their community through photography. Participants also spent ten weeks on 
environmental projects, such as cleaning and planting a nearby community garden. 

Keeping school-day conflicts from spilling over into the after-school project is an 
important part of staff training. Two activities, in particular, have been instrumental. Ramapo’s 
Youth Workers as Leaders teaches strategies to staff for dealing with behavior problems, 
especially among students with special educational or social needs. In addition, the Crown 
Heights Mediation Center conducts conflict resolution training using simulations in which staff 
practice diffusing conflicts before they get out of hand. 
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SAMUEL FIELD Y 
Bayside, Queens 

Project Overview 

Project Start Date: 1999 
Grades Served: K-5 
2004-05 Budget: $216,000 
Enrollment: 157 
Attendance Rate: 65% 

Samuel Field Y takes a holistic approach to creating an environment that nurtures 
participants’ psychosocial development. Staff use structured curricula but employ them flexibly 
so that activities suit participants’ developmental levels. According to site coordinator Iris Shaw, 
by the end of the school day, students are “saturated” and “don’t have any more to give,” so the 
project aims to foster strong relationships between youth and staff as preparation for participants’ 
eventual transition to middle school. 

Youth participate in three activities each day, including structured homework help and 
project-based academic enrichment. On Friday Club Days, participants select from a menu of 
activities, including dance, arts and crafts. Girl Scouts, cooking, movies, board games, and 
Comic Book Club, in which they write and draw their own theme-based comics on leadership, 
the environment, and similar topics. They also take field trips. 

Ms. Shaw looks for young leaders who have a “true calling to work with children,” and 
relies on training and mentoring, along with the staff’s own hard work, to develop effective after- 
school workers. Staff development takes place all year long as younger, inexperienced leaders 
pair with more seasoned ones. In addition to their on-site training, staff participate in TASC- 
sponsored training programs in their areas of interest. 

Formal job evaluations have given way to more informal but continuous conversations 
about performance. The site coordinator routinely gives staff verbal and written feedback and 
asks staff to respond to her comments. More formal staff meetings take place monthly. 

Recent years have brought challenges. The host school is pressing the project to offer 
more academic and fewer developmental activities. Declining school enrollment has reduced the 
pool of eligible after-school participants. The project is addressing these challenges through 
increased dialogue with the host school and stepped-up outreach and recruitment. 



Population Served 

Eligible for Free/Reduced Price Funch: 52% 
Recent Immigrants : 1 8 % 

English Fanguage Feamers: 10% 

Eligible for Special Education Services: 8% 
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CHAMPIONS CLUBS OF 

SPORTS AND ARTS IN SCHOOLS FOUNDATION 

Astoria, Queens 



Project Overview 

Project Start Date: 1999 
Grades Served: K-8 
2004-05 Budget: $293,000 
Enrollment: 299 
Attendance Rate: 64% 



Population Served 

Eligible for Eree/Reduced Price Eunch: 83% 
Recent Immigrants : 1 3 % 

English Eanguage Eeamers: 26% 

Eligible for Special Education Services: 1% 



Although Champion’s Club began in 1999 across a number of New York City schools 
and emphasized sports, its arts activities — and most notably its musical productions — now draw 
increasing attention, especially as New York City schools have curtailed in-school arts programs. 

Sports and arts activities wrap around structured homework, which, to minimize 
distraction, takes place four days a week on a separate floor. Half the groups start the afternoon 
with homework, while the other half finishes up that way. This enables homework sessions to 
serve no more than 10 participants per group. Activities in the arts and in sports reflect the 
talents of staff and interests of participants, and are designed to give every interested youngster a 
chance to explore new avenues for success. The project’s musical performances give 
participants a chance to sing and dance to Broadway show tunes with complex choreography and 
to create elaborate set designs. Sports offerings are aligned with professional sports seasons, so 
that project staff can take participants to related sports events. Inter-mural sports activities, 
together with community-wide competitions, motivate participants and encourage family and 
community support. 

The community joins the after-school project each year on Make a Difference Day. 
Participants donate toys or clothing to a nearby shelter for homeless families, or take part in food 
drives, penny drives, charity Olympics, and walk-a-thons. These community events usually 
involve parents in planning and in directing activities. 

The project is staffed by a stable mix of certified teachers, adult leaders (many who have 
specialized sports or arts expertise), AmeriCorps members, and high school students. About one 
third of the staff has been with the project for four or more years; five staff have been with the 
project since its inception. Using internal staff rather than contracting with specialists from 
outside helps keep costs down and gives staff opportunities to explore and stretch their own 
talents in areas of interest. 

Mentoring is a most powerful training and professional development strategy. Site 
coordinator Jan Vazquez pairs new staff “with someone who knows the ropes,” so veteran staff 
can coach newer ones. Site-based training programs cover topics such as creating quality 
activities, supporting participants who are experiencing family stress, and minimizing classroom 
behavior problems. 
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STANLEY M. ISAACS NEIGHBORHOOD CENTER 
AFTER-SCHOOL PROJECT 
Central Park East, Manhattan 



Project Overview 

Project Start Date: 1999 
Grades Served: K-5 
2004-05 Budget: $212,000 
Enrollment: 193 
Attendance Rate : 7 5 % 



Population Served 

Eligible for Eree-Reduced Price Eunch: 99% 
Recent Immigrants: 4% 

English Eanguage Eeamers: 8% 

Eligible for Special Education Services: 7% 



The building in which this after-school project is located houses two very different 
schools: one is a high- achieving magnet school that admits students through an application 
process, the other is a low-performing school with many poor students. After-school services are 
open to everyone. The center also operates a Beacon program from 6 to 10 p.m. in the same 
space. Some students participate in both the TASC after-school project and the Beacon program. 

Site coordinator Autumn King says about the after-school project operated by the Stanley 
M. Isaacs Neighborhood Center, “It’s about empowering [youth] and letting them understand 
what they can do and who they are.” Staff urge youth to set goals and maintain high 
expectations for behavior. At the start of the year, youth make “promise wheels” that spell out 
behaviors they promise to use. They set their own consequences for breaking these promises. 

The project uses the center’s curriculum. Helping Everyone Live Peacefully or HELP, to 
discourage teasing and bullying. 

Aside from doing homework, students participate in sports and physical exercise, arts, 
and enrichment activities such as chess, drumming, and dance, all led by specialists in their 
respective fields. Monthly themes, ranging from historical events to personal health, unify the 
program. Eor example, a science activity highlighted a woman scientist during Women’s 
History Month. Eriday clubs break from established curricula and allow staff to create their own 
activities that they think participants will enjoy. When staff design activities, the site coordinator 
asks them “what it satisfies” within literacy, math or history so that the after-school project 
supports school-day learning. 

Group leaders, all high school and college students, work with the specialists and with a 
master teacher who works in the regular school during the day and serves as a liaison between 
the school and the after-school project. The project places a great deal of emphasis on staff 
development for its young staff. Two staff development meetings take place each week, often 
including workshops or talks by outside speakers. The Isaacs Center provides scholarships for 
training and education, and often offers other career opportunities for more experienced group 
leaders. 
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WOMEN’S HOUSING AND ECONOMIC DEVELOPMENT 
CORPORATION AFTER-SCHOOL ENRICHMENT PROJECT 

South Bronx 



Project Overview 

Project Start Date: 1999 
Grades Served: K-8 
2003-04 Budget: $371,000 
Enrollment: 409 
Attendance Rate: 76% 



Population Served 

Eligible for Eree/Reduced Price Eunch: 100% 
Recent Immigrants: 5% 

English Eanguage Eeamers: 32% 

Eligible for Special Education Services: 4% 



Participants in this project are organized into three cluster groups: grades K-2, 3-5, and 
6-8. At least one adult and an assistant lead each group of 15 to 20 participants, with the help of 
specialists — often professionals from these fields — who connect the school-day and after-school 
programs and conduct most arts, academic enrichment, and sports activities. 

Helping participants to make healthy choices is a dominant theme in this after-school 
program. Middle-grade students, in particular, have an added focus on building self-esteem and 
relationships, avoiding risk-taking behaviors, and conflict resolution. A seventh- and eighth- 
grade “daily living skills” club, directed by social workers, works to prevent teenage pregnancy, 
using tools of dialogue, information about pregnancy prevention and healthy life styles, and 
action planning. In addition, drama classes involve participants in Teen Advocates Theater, 
which uses peer educators in interactive theatrical productions to educate young people on sexual 
health and pregnancy prevention. 

High achieving middle-school students in the school’s Education Career Guidance Center 
work with teaching specialists after school to strengthen their academic skills and to prepare for 
the high school application and admission process. Activities involve research to select high 
schools that are academically strong in the areas in which youth have career interests. 

The after-school project serves the entire family, inviting parents and guardians to 
participate in WHEDCO services such as homeless prevention, welfare-to-work job training, 
family day care, fitness and nutrition, and a complete range of social and clinical services. 
Through two full-time WHEDCO social workers who work in the school, teachers may refer 
students for services who are not enrolled in the after-school project. 

The project’s parent coordinator encourages parents to volunteer and participate in 
project events, which occur several times a year. The project surveys parents annually to assess 
their satisfaction and to generate recommendations for improvements. A small parent council 
convenes periodically to advise leaders about successes and needs from the community’s 
perspective. 
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